Test regional response-rate differences rigorously

Read the full interview experience this question came from →

Quick Overview

This question evaluates applied statistical modeling and causal-inference competencies, including adjustment for confounding, analysis of proportion/count outcomes, variance estimation under clustering, multiple-comparison control, diagnostic detection of Simpson’s paradox, and clear effect-size reporting; it is categorized under Statistics & Math and emphasizes practical application with necessary conceptual understanding. Such problems are commonly asked to assess the ability to justify and interpret comparative analyses that account for covariate mix, temporal structure, and multiple-testing concerns, and this summary is in English.

Test regional response-rate differences rigorously

Company: Thumbtack

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Onsite

Using the same job-level data (job_category, region, invitations_sent, provider_responses, created_at), determine whether response rates differ by region after adjusting for job_category mix and time. Specify and justify a statistical approach (e.g., logistic regression with response as successes/tries, or a mixed-effects model with region random effects). Include: (1) model formula, link function, and any fixed effects (job_category, month) and interactions; (2) how you will cluster or regularize standard errors; (3) a multiple-comparison strategy across many regions (e.g., Holm–Bonferroni or Benjamini–Hochberg on region coefficients); (4) a diagnostic to detect Simpson’s paradox driven by job_category mix; and (5) how you would report effect sizes and practical significance, not just p-values.

Overview: This question evaluates applied statistical modeling and causal-inference competencies, including adjustment for confounding, analysis of proportion/count outcomes, variance estimation under clustering, multiple-comparison control, diagnostic detection of Simpson’s paradox, and clear effect-size reporting; it is categorized under Statistics & Math and emphasizes practical application with necessary conceptual understanding. Such problems are commonly asked to assess the ability to justify and interpret comparative analyses that account for covariate mix, temporal structure, and multiple-testing concerns, and this summary is in English.

Read the full Thumbtack Data Scientist interview experience this question came from

|Home/Statistics & Math/Thumbtack
Thumbtack logo
Thumbtack
Oct 13, 2025
mediumData ScientistOnsiteStatistics & Math
13
0

Goal

Assess whether provider response rates differ by region after adjusting for job category mix and time.

Data

You have job-level observations with the following fields:

  • job_category (categorical)
  • region (categorical)
  • invitations_sent (integer ≥ 0)
  • provider_responses (integer ≥ 0, ≤ invitations_sent)
  • created_at (timestamp)

Interpretation: For each job, invitations_sent providers were invited and provider_responses of them responded.

Task

Propose and justify a statistical approach to estimate regional differences in response rates while controlling for job_category and time. Include:

  1. Model specification: family/link, explicit formula, fixed effects for job_category and time (e.g., month), and any interactions you recommend.
  2. How you will compute or regularize standard errors (e.g., clustering/robust variance).
  3. A multiple-comparison strategy across many regions (e.g., Holm–Bonferroni or Benjamini–Hochberg on region effects).
  4. A diagnostic for Simpson’s paradox driven by job_category mix.
  5. How you will report effect sizes and practical significance, not just p-values.

State any minimal assumptions you need (e.g., how you define time buckets from created_at).

Loading comments...

Browse More Questions

More Statistics & Math•More Thumbtack•More Data Scientist•Thumbtack Data Scientist•Thumbtack Statistics & Math•Data Scientist Statistics & Math