The 6-Step ML SD Framework

Use six steps to organize an ML system-design answer: scope, data, model, evaluation, serving, and monitoring. The order helps expose dependencies, while the prompt’s constraints determine where to spend time and when to revisit an earlier choice.

An interviewer asks, “Design a fraud detection system for a payments platform.” You have 45 minutes to turn that prompt into a coherent design. Without a plan, many candidates fall into a common failure mode: they jump between model architectures, data pipelines, and evaluation metrics without a clear sequence, making the design hard to evaluate. The fix is not necessarily more ML knowledge; it is a structured approach.

The 6-step ML System Design framework is a checklist for covering the design. It does not guarantee completeness, but it gives you and the interviewer a shared route through these topics:

• Problem clarification and scope

• Data strategy

• Model design

• Evaluation (offline and online)

• Serving and deployment

• Monitoring and iteration

Each step builds on the one before it, and together they form a repeatable mental model you can apply whether the prompt is about recommendation systems, ad-click prediction, or retrieval-augmented generation.

Use the sequence as a starting point and explain any change of order. The rest of the lesson describes what each step contributes and when a constraint should move a discussion earlier.

The following diagram illustrates how the six steps connect as a pipeline, with a feedback loop from monitoring back to data strategy that reflects the iterative nature of production ML systems.

Steps 1 to 3: From ambiguity to architecture

The first part of the framework turns a vague system-design prompt into a concrete ML model design. Each step narrows the solution space before you make implementation decisions, so later decisions are based on explicit requirements and constraints.

Step 1: Problem clarification and scope

Every ML system design prompt is intentionally ambiguous. “Design a recommendation system” could mean home page feed ranking, email digest personalization, or related-item suggestions. Your first job is to convert that ambiguity into a precise problem statement.

Ask about the target user, the business objective, scale expectations, and latency constraints. If the interviewer says “recommend videos,” clarify whether the goal is to maximize watch time, increase content diversity, or reduce churn. These distinctions change everything downstream, from the loss function to the serving architecture.

Practical tip: Spend 3 to 5 minutes on scoping. Use that time to agree on the prediction target and constraints so later choices answer the same problem.

Step 2: Data strategy

Models are only as good as the data that feeds them. This step belongs immediately after scoping because every model decision, from architecture to training procedure, depends on what signals you can actually obtain.

Cover these areas in your answer:

• Data sources: Identify available logs, user profiles, contextual signals, and any third-party data. For the fraud-detection example, transaction logs, device fingerprints, and merchant history are all potential sources.

• Labeling approach: Distinguish direct judgments, such as ratings or reports, from behavioral signals such as clicks. Both can be noisy or selective. For fraud, explain how a report becomes a confirmed outcome and when that label is available.

• Feature engineering: Decide which raw signals become model features. Aggregations like “number of transactions in the last hour” or embeddings of categorical fields, such as merchant category, are typical choices.

• Data freshness: Determine whether features need real-time computation or whether daily batch updates suffice. A fraud system likely needs near-real-time features, while a weekly email recommender does not.

Addressing data quality here prevents cascading problems in model design and evaluation.

Step 3: Model design

With a clear problem and a data strategy in hand, you can now reason about architectures. Compare plausible models on the data, quality, and serving requirements you established.

Use a YouTube-style recommendation pipeline as a grounding example. In a common design, candidate generation uses a two-tower model that embeds users and videos in the same vector space, enabling approximate nearest-neighbor search to retrieve likely relevant videos. The ranking stage can then use a deeper model with richer features to score the smaller set returned by candidate generation. If latency is tight or the available training data is mostly structured/tabular, a gradient-boosted tree can be a practical ranking baseline.

Attention: Start with a feasible baseline, then explain what a more complex architecture could improve and how you would test that improvement.

The table below summarizes all six steps, what to cover in each, why each occupies its position, and what interviewers are listening for.

ML System Design Interview Framework

With the first three steps covered, the next section walks through how you validate, deploy, and maintain the system you have designed.

Steps 4 to 6: From validation to production

The second part of the framework connects the proposed model design to a production system that can serve users reliably at scale. Reserve time for evaluation, deployment, and monitoring so the design explains how the model will operate after launch.

Step 4: Evaluation

Offline evaluation

Offline metrics measure model quality on held-out data before anything reaches production. The choice of metric depends on the task. Classification problems like fraud detection use precision, recall, and AUC (Area under the ROC curve). Ranking problems like search or recommendations use NDCG (Normalized discounted cumulative gain).

Online evaluation

Offline metrics alone are insufficient. A model that maximizes click-through rate in offline tests may hurt long-term user retention, which is the metric the business actually cares about. Online evaluation through A/B tests and interleaving experiments measures real-world impact.

Note: Interviewers specifically probe whether you understand the gap between offline metrics and business objectives. It is not enough to say, “We evaluate the model with offline AUC and validate it with an online A/B test.” Explain how you would monitor for metric divergence, such as higher offline AUC paired with lower retention, engagement, or long-term satisfaction in the online test. Then describe how you would investigate the gap, check segment-level impacts, review labels and objectives, and roll back or adjust the model if the product metrics regress.

The comparison tests whether the chosen proxy still serves the product objective.

Step 5: Serving and deployment

A model that cannot meet production constraints is a model that never ships. This step covers the infrastructure that bridges training to serving.

• Serving pattern: A live arrival-time request may need real-time inference, while a scheduled recommendation email can use batch predictions. Treat a sub-one-hundred-millisecond budget as a possible requirement to clarify, not a verified universal benchmark for Uber.

• Scalability: Horizontal scaling distributes load across replicas. Model distillation is common when a large ranking model must serve millions of queries per second.

• Feature stores: Shared, versioned features can reduce duplicated work. Verify point-in-time joins, defaults, transformations, and freshness; a store alone does not eliminate training-serving skew.

• Canary deployments: Rolling out a new model to a small percentage of traffic before full deployment catches production issues early without exposing all users to risk.

Connect each serving decision to a measurable operating requirement.

Step 6: Monitoring and iteration

Inadequate monitoring is a primary failure mode in production ML systems. Models degrade silently as user behavior shifts, data distributions change, or upstream pipelines break.

Cover concept drift, where the feature-label relationship changes, and data drift, where input distributions change. Explain the alerts, diagnosis, evaluated retraining, and rollback paths. Shadow scoring runs a candidate alongside the live model without using its results for customer decisions.

This step closes the loop. The dashed arrow in the pipeline diagram points from monitoring back to data strategy because monitoring insights, such as discovering that a new fraud pattern has emerged, feed directly into the next iteration of data collection and feature engineering.

Practical tip: If you propose shadow scoring or automated retraining, explain the trigger, comparison data, and promotion gate. A schedule alone does not establish that a replacement model is safe to serve.

The following quiz tests whether you can place a real design decision in the correct framework step:

When to deviate from the canonical order

The six steps are a default sequence, not a mandate. Experienced candidates sometimes reorder or merge steps, and doing so deliberately can demonstrate stronger judgment than following the framework rigidly.

Consider this prompt: “Design a real-time ad-click prediction system that serves 1 million QPS.” In this case, serving constraints drive the design. Start by scoping the problem briefly, then cover serving and deployment constraints before selecting the model architecture. Latency, throughput, and cost constraints at that scale eliminate entire classes of models. For example, a transformer ensemble with 500 ms inference latency is unlikely to fit before you even evaluate the rest of the architecture.

A different scenario arises with generative AI prompts such as “design a retrieval-augmented generation system.” Evaluation for generative outputs is unusually complex because there is no single ground-truth answer. Discussing evaluation criteria early, before choosing an architecture, helps align you and the interviewer on what “good” means.

The key principle in both cases is communication. Always signal your deviation explicitly. Saying something like “I’d like to discuss serving constraints first because they’ll heavily constrain our model choices. Does that work for you?” makes the reason for changing the sequence explicit.

A useful change of order addresses a specific dependency. For the high-QPS example, the serving budget constrains model choice; for generation, the evaluation criteria clarify what the output must achieve.

Bringing the framework together

The 6-step framework is a communication tool as much as a thinking tool. It gives the interviewer a mental map of where you are in your answer and where you are headed. Problem clarification converts ambiguity into a precise scope. Data strategy identifies the signals and labels your model will consume. Model design selects an architecture justified by trade-offs. Evaluation validates performance offline and online against business objectives. Serving and deployment bridges the model to production under real-world constraints. Monitoring and iteration ensures the system stays healthy long after launch.

Practice by tracing the opening fraud prompt through all six steps. Identify one decision in each step that depends on an earlier answer. The next lesson assigns a time budget so the final evaluation, serving, and monitoring discussions remain part of the answer.