Diagnose data quality and pipeline performance issues

Scenario

You are interviewing for a Data Solutions Architect role. A customer is using a cloud data platform (e.g., Databricks on AWS/Azure/GCP) and reports:

• Data quality issues (incorrect/missing/duplicated records, inconsistent definitions)
• Performance issues (slow ETL/ELT pipelines, long query times, high compute cost)

They ask: “We’re struggling with data quality and performance—how would you approach this?”

Tasks

1. Discovery & scoping: What questions do you ask to clarify the problem and constraints?
2. Define success: What metrics would you use for (a) data quality and (b) performance/cost? Include primary metrics and guardrails.
3. Diagnosis plan: Describe a step-by-step approach to identify root causes (data sources, pipeline stages, storage layer, compute, governance).
4. Solution proposal: Propose concrete technical and process changes to:
   • Improve data quality (validation, monitoring, ownership, SLAs)
   • Improve performance (storage layout, compute configuration, pipeline design)
5. Concept check: Explain the differences between a data lake and a data warehouse , and where a lakehouse fits.
6. Cloud considerations: What cloud concepts commonly matter in these engagements (e.g., security/IAM, networking, storage, encryption, cost)?

Deliverable

Provide a structured plan you could present to the customer (bullets are fine), including short-term mitigations and longer-term architecture/process recommendations.