Xometry Senior+ Machine Learning Engineer Interview Experience — A Deploy-Under-200ms Deep Dive

Company: Xometry (March)
Round: Round 1
Position: Lead ML

Questions/format:

1. Walk through your background and your projects in detail
2. The DS team handed you a model — if you had to deploy it, the requirement is that a single inference call has to come in under 200ms

1) Key points for talking about project background

• Project goal: the business problem you're solving, and the evaluation metrics (e.g. accuracy, recall, latency, cost, etc.)
• Data: data source, scale, feature types, class imbalance and how you handled it
• Model architecture: the model you used (e.g. XGBoost/CatBoost/LightGBM / CNN / Transformer, etc.), inputs/outputs, key hyperparameters
• Training and validation: train/validation/test split, cross-validation, metrics (AUC/F1/MAE, etc.), baseline comparison
• Engineering and deployment: preprocessing pipeline, feature engineering, online/offline features, model monitoring, deployment mode (batch/online)
• Your contribution: which module you owned, the hard parts you ran into, how you solved them, quantitative results (numbers that actually make the point)
• What you learned and what you'd improve

2) How to think about and answer "deploy the model with a <200ms requirement"

• Clarify the constraints first (you should proactively confirm these):

• Is the 200ms p50/p95/p99, or average latency?

• Expected concurrency (QPS), size of the input data (image/text/numeric), and whether preprocessing is included.

• Acceptable accuracy-drop threshold, available hardware (CPU/GPU/TPU), and memory limits.

• Optimizations and approaches (from simple to more complex):

1. Quantization: FP16/INT8 (static/dynamic/quantization-aware training) — usually cuts latency and memory significantly.
2. Distillation: use a smaller model to approximate the performance of the larger one.
3. Pruning and sparsification: reduce the parameter count.
4. Switch to a lighter model architecture: MobileNet, EfficientNet-lite, TinyBERT/DistilBERT, etc.
5. Model conversion and acceleration libraries: ONNX -> ONNX Runtime / TensorRT / OpenVINO / TorchScript / TVM.
6. Serving and engineering optimizations: Triton Inference Server, gRPC/HTTP load testing, thread/process tuning, batching strategy (watch out for single-request latency), connection pooling, warm-up.
7. Precomputation and caching: cache reusable features/intermediate computations to cut online compute cost.
8. Parallelism and distribution: if the input can be processed in parallel, consider pipeline parallelism or model sharding (watch the network overhead).
9. Hardware choice: a high-clock-speed CPU, a GPU with TensorRT, NVIDIA Tensor Cores, or a dedicated inference accelerator.
10. Tooling: use a profiler (perf/pyinstrument/nsight/torch.profiler) to locate the bottleneck (preprocessing/model/IO).