Time Series Modeling And Forecasting

What's being tested

The interviewer is testing whether you can turn a temporal prediction problem into a reliable machine learning system, not merely select an algorithm. You should be able to define the forecasting unit, horizon, granularity, available-at-prediction-time features, validation scheme, serving path, and operational metrics. Microsoft cares because forecasts often drive capacity planning, ranking, recommendations, inventory, and cloud-resource allocation; leakage, stale features, or poorly monitored models can create expensive systematic errors.

Core knowledge

• Forecasting contract: Define the entity or series, timestamp semantics, frequency, forecast horizon, prediction quantiles, retraining cadence, and latency SLA. “Predict next week” is incomplete unless you specify whether predictions are hourly, daily, per region, and generated once or continuously.

• Temporal leakage: Every feature must satisfy $t_\text{feature} \leq t_\text{prediction}$. Random train/test splits leak future behavior through rolling aggregates, normalization, backfilled labels, or revised external data; use point-in-time feature retrieval from a feature store.

• Backtesting: Use rolling-origin evaluation: train through time $t$, forecast the next horizon, advance the cutoff, and repeat. Preserve realistic gaps for label availability and feature computation; the final test window should remain untouched until model selection is complete.

• Baselines: Always compare against seasonal naive forecasts, such as $\hat{y}_{t+h}=y_{t+h-m}$ for seasonal period $m$, and simple moving-average or exponential-smoothing models. A complex model that does not beat these baselines consistently is not production-ready.

• Model families: ARIMA/SARIMA captures autoregressive, differenced, and seasonal structure; ETS models level, trend, and seasonality; tree models such as LightGBM use lag and calendar features; global neural models such as DeepAR share information across many related series.

• Feature construction: Common features include lags, rolling means and standard deviations, calendar indicators, promotions, holidays, and known-future covariates. Compute windows using only historical rows, define behavior for missing lags, and avoid using a target-derived feature whose availability differs online.

• Metrics: Use MAE for interpretable absolute error, RMSE when large misses deserve extra weight, and WAPE for aggregate business error when denominators are meaningful. For probabilistic forecasts, use pinball loss for quantiles and weighted interval score for calibrated intervals.

• Scale and aggregation: For millions of series, train a global model or cluster series rather than fitting one model per entity. Reconcile hierarchical forecasts when regional predictions must sum to a corporate total; independently accurate levels can otherwise contradict one another.

• Probabilistic output: Point forecasts hide uncertainty. Predict quantiles such as $P_{50}$, $P_{90}$, or a full distribution, then test coverage: approximately 90% of outcomes should fall below a calibrated $P_{90}$ forecast. Check both calibration and sharpness.

• Serving architecture: Batch forecasts are appropriate for daily planning; online inference is appropriate when forecasts depend on current state. Keep feature computation, model versioning, input schemas, and transformations consistent between training and serving, with fallback to a seasonal-naive model.

• Monitoring: Track data freshness, missingness, feature distributions, forecast volume, latency, error by horizon and segment, residual autocorrelation, and prediction-interval coverage. Actual-error monitoring is delayed, so use input and prediction drift as leading indicators rather than treating drift as proof of model failure.

• Operational edge cases: Handle daylight-saving transitions, timezone boundaries, series with short histories, zero-inflated or intermittent demand, new entities, outliers, delayed labels, and changed business definitions. For intermittent demand, Croston-style methods or two-stage occurrence/size models may outperform ordinary regression.

Worked example — Design a production time-series forecasting system

A strong candidate would first clarify the exact series, forecast horizon, frequency, number of entities, whether future promotions or capacity plans are known, and whether predictions are batch or online. I would state an assumption such as: “We need daily forecasts for 100,000 regional-product series, seven days ahead, refreshed each night, with $P_{50}$ and $P_{90}$ outputs.” I would organize the answer around four pillars: point-in-time data and feature generation, rolling-origin evaluation against seasonal-naive baselines, model and probabilistic-output selection, and deployment plus monitoring. I would begin with lag, rolling, calendar, and known-future features in a global LightGBM model, then compare it with a global neural model if scale and cross-series patterns justify the complexity. The explicit tradeoff is per-series models versus a global model: per-series models are easier to reason about for stable long histories, while a global model shares statistical strength and handles many short series more efficiently. I would describe a batch pipeline that materializes features, validates schemas and freshness, scores the versioned model, stores forecasts and metadata, and exposes them to downstream consumers. Evaluation would report aggregate and per-segment MAE or WAPE, horizon-specific error, quantile loss, and $P_{90}$ coverage rather than one average number. I would close by saying that, with more time, I would investigate hierarchical reconciliation, cold-start handling, backtesting under historical outages, and rollback behavior when actuals or covariates arrive late.

A second angle — Evaluate a multi-step demand forecasting model

For a question focused on evaluation, the framing changes from architecture to whether the offline result predicts production behavior. I would compare direct, recursive, and multi-output forecasting strategies because recursive error can compound across a seven-day horizon. I would use rolling backtests with the same forecast cutoffs and feature availability as production, stratify results by series volume and intermittency, and compare against seasonal-naive forecasts. I would also test interval calibration and business-weighted losses, since a model with lower average MAE may be worse when under-forecasting causes capacity shortages. Finally, I would reserve the latest time block for an untouched final check and use a shadow or canary deployment before replacing the existing model.

Common pitfalls

Pitfall: Randomly splitting rows and reporting a strong validation score is a classic leakage failure. Use chronological, rolling-origin splits and prove that every feature would have existed when the prediction was generated.

Pitfall: Saying “I would use an LSTM” without defining the target, horizon, baseline, or evaluation metric sounds like algorithm shopping. Lead with the forecasting contract and a simple deployable baseline, then justify additional complexity with measured gains.

Pitfall: Monitoring only RMSE after deployment misses stale features, broken calendars, cold-start entities, and poorly calibrated uncertainty. Describe operational checks, segment-level error, delayed-label handling, and a safe fallback path.

Connections

An interviewer may pivot to feature-store point-in-time correctness, concept drift and retraining, probabilistic calibration, or hierarchical reconciliation. They may also connect forecasting to online experimentation when predictions influence a capacity or product decision.

Further reading

• Forecasting: Principles and Practice — Hyndman and Athanasopoulos’s practical treatment of baselines, backtesting, hierarchical forecasting, and probabilistic evaluation.

• DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks — Introduces global probabilistic forecasting across related time series.

• Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting — Covers multi-horizon prediction with static, observed, and known-future covariates.