How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a medium difficulty Machine Learning question, commonly asked during Onsite rounds at Thumbtack.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Thumbtack during technical interviews.

Forecast response-rate trends with backtesting | Thumbtack Interview Question

Quick Overview

This question evaluates proficiency in time-series forecasting and model validation, including feature engineering, model selection, rolling-origin backtesting, evaluation metrics, and change-point detection for response-rate prediction at the job_category-week level.

Forecasting Response Rate by Job Category and Week

Context

You are given weekly marketplace data with invitations and responses by job_category and region. Define response rate at the job_category-week level as:

response_rate = responses / invitations

Assume invitations are the denominator of interest and weeks without invitations are excluded or imputed carefully. The goal is to forecast the next 4 weeks of response rates for each job_category and describe the modeling approach.

Task

Aggregate to job_category-week and build a model to predict the next-4-week response rates for each job_category. Describe the following:

Feature set: calendar dummies, region mix, invitations per job, seasonality terms, holiday flags.
Candidate models and why: SARIMAX with exogenous regressors vs. gradient boosted trees on lag features.
Backtesting protocol: rolling-origin evaluation with a 4-week horizon.
Metrics: sMAPE for accuracy, calibration of predictive intervals.
Change-point detection to guard against structural breaks, such as policy changes.
How to use the forecasts for staffing or budget decisions.

Quick Overview

Context

You are given weekly marketplace data with invitations and responses by job_category and region. Define response rate at the job_category-week level as:

response_rate = responses / invitations

Task

Aggregate to job_category-week and build a model to predict the next-4-week response rates for each job_category. Describe the following:

Feature set: calendar dummies, region mix, invitations per job, seasonality terms, holiday flags.

Candidate models and why: SARIMAX with exogenous regressors vs. gradient boosted trees on lag features.

Backtesting protocol: rolling-origin evaluation with a 4-week horizon.

Metrics: sMAPE for accuracy, calibration of predictive intervals.

Change-point detection to guard against structural breaks, such as policy changes.

How to use the forecasts for staffing or budget decisions.

Forecast response-rate trends with backtesting

Quick Overview

Forecast response-rate trends with backtesting

Forecasting Response Rate by Job Category and Week

Context

Task

Write your answer

Forecast response-rate trends with backtesting

Quick Overview

Forecast response-rate trends with backtesting

Forecasting Response Rate by Job Category and Week

Context

Task

Write your answer