As a Data Scientist at Major League Baseball (MLB), you sit at the intersection of high-stakes sports analytics and massive-scale digital product development. This role is pivotal in transforming vast streams of live game data and fan interaction metrics into actionable intelligence. Your work directly influences how millions of fans experience baseball, ranging from personalized content recommendations on MLB.com to the optimization of digital streaming infrastructure.
You will operate in a complex environment where technical precision meets the fast-paced nature of the sports industry. Whether you are building predictive models for engagement or analyzing user behavior across the MLB digital ecosystem, your contributions are essential to maintaining the league’s position as a leader in sports technology. The role requires a blend of rigorous statistical analysis and the ability to translate complex data findings into clear, strategic narratives for non-technical stakeholders.
Preparation focus
editorialNo round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.
What to demonstrate
- Breadth across SQL, experimentation and product reasoning
- Ability to state assumptions before choosing a method
How to prepare
- Drill the practice exercises below and time yourself
- Prepare three quantified stories about decisions you drove
PracHub editorial advice for the preparation topics above.
Counting on an identity key that changes underneath the metric
visitor_id is per browser and per device, and it resets on cookie clearance, private browsing and platform privacy changes, so the distinct-visitor count drifts upward for reasons unrelated to reach. Any rate with visitors in the denominator therefore decays over time even when behaviour is constant, and any rate with visitors in the numerator inflates. The stitching at signup makes it worse in both directions: a user who signed up on mobile and returns on desktop is two visitors and one user, while a shared device is one visitor and several users. Decide which key each metric is counted on, write it into the definition, and when comparing a period before and after a platform privacy change, expect a level shift in every visitor-keyed metric and do not attribute it to the product.
Comparing cohort retention curves of different maturities, or building the curve from users who are still present
A cohort four weeks old has no week-8 value, so an average taken across cohorts silently drops young cohorts from the later columns and keeps them in the earlier ones. The curve then bends upward at the tail, and the reading that 'retention is improving over time' is an artefact of which cohorts survived to be measured. The same error appears in the denominator when retention is computed over users active in the current period rather than over the full original cohort, which conditions on survival and guarantees a flattering number. The fix is a triangle: fix the cohort at signup, bound every window on both sides, and only compare cells where every cohort has had the full elapsed time, publishing the rest as blank rather than as a partial average.
Answering a product-sense question with a list of features
Answer with a decision and the measurement that would settle it: the hypothesis, the primary metric, the guardrails, and the result that would make you not ship. A feature brainstorm cannot be wrong, which is exactly why it earns no points.
Ignoring interference between units in a marketplace experiment
Ask whether one unit's treatment can change another unit's outcome through shared inventory, a matching pool, a social graph or a common budget. Where it can, randomise at a level that contains the spillover, such as region or time slice, and say explicitly what that costs you in statistical power.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Simulate the false positive cost of repeated peeking
Quantify the cost of peeking. Simulate a two-arm experiment with no true effect: each arm accumulates Bernoulli conversions at a base rate of 0.10 up to 40,000 units per arm. Run a two-sided two-proportion z-test at alpha 0.05 at ten equally spaced interim points, and record whether the test ever crossed. Report the false positive rate over at least 10,000 replications, alongside the rate for a single look at the final sample only. Use a fixed seed and report a Monte Carlo standard error on both figures.
Approach
- Generate each replication as two cumulative sums of Bernoulli draws, then read the interim points off the cumulative arrays. Regenerating data at each look would make the looks independent, which destroys exactly the dependence the exercise is about: later looks share data with earlier ones.
- Use the pooled-variance two-proportion z: p_pool = (x1+x2)/(n1+n2), z = (p1-p2) / sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2)), reject when |z| > 1.96. State that the normal approximation is fine here because the smallest look has roughly 400 expected conversions per arm.
- Vectorise across replications rather than looping: draw a (reps, n) array of uniforms, threshold at 0.10, cumsum along axis 1 and slice the ten look indices. A per-replication loop at 10,000 by 40,000 is unnecessarily slow.
- Record the any-cross indicator per replication, take the mean, and compute the Monte Carlo standard error as sqrt(p*(1-p)/reps) so the reported figure comes with its own precision.
- Report the single-look rate in the same run as a control. If it does not land near 0.05, the bug is in the test statistic and not in the peeking argument.
Follow-up
- Re-run with 40 looks instead of 10. Why does the curve flatten rather than continue rising linearly?
- Among the replications that crossed, what is the mean observed lift, and why is it not zero?
- What does an O'Brien-Fleming boundary or an always-valid confidence sequence change about this simulation, and what does each cost in power?
Cluster bootstrap for a per-session rate randomised on users
An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.
Approach
- Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
- Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
- Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
- Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
- Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.
Worked solution 35 min
- per_user = df.groupby(['variant','user_id'])['converted'].agg(['sum','size']); split into two arrays per arm.
- point = (t_sum.sum() / t_n.sum()) - (c_sum.sum() / c_n.sum()).
- For b in range(B): idx = rng.integers(0, len(t_sum), len(t_sum)); ratio_t = t_sum[idx].sum() / t_n[idx].sum(); same for control; store the difference. Vectorise by drawing a (B, n) index matrix if memory allows.
- ci = np.percentile(diffs, [2.5, 97.5]); naive_se = sqrt(p_t*(1-p_t)/n_sessions_t + p_c*(1-p_c)/n_sessions_c); naive_ci = point +/- 1.96*naive_se.
- width_ratio = (ci[1]-ci[0]) / (naive_ci[1]-naive_ci[0]).
Follow-up
- Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
- Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
- Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?
Split a pooled conversion drop into rate and mix
You have weekly visit-to-signup counts by segment: a DataFrame with week, device_type, referrer_channel, visitors and signups. The pooled rate fell 0.84 percentage points between two consecutive weeks while several individual segments rose. Write a function that, for a caller-supplied list of segment columns, splits the pooled change into a rate effect, a mix effect and an interaction term that sum exactly to the observed change. Return those three scalars plus a per-segment contribution table sorted by absolute contribution, so the largest single driver can be named.
Approach
- State the algebra before coding: the pooled rate is r = sum over segments of w_s * r_s, with w_s the segment's share of the denominator. Then r1 - r0 decomposes exactly into sum(w_s0 * (r_s1 - r_s0)) for rate, sum((w_s1 - w_s0) * r_s0) for mix, and sum((w_s1 - w_s0) * (r_s1 - r_s0)) for interaction. The identity is per-segment, so it holds for any numbers you put in the four slots.
- Pivot both weeks onto a common segment index with an outer join so a segment that appeared or vanished is kept rather than dropped, then decide what rate to give a segment with no visitors in one of the weeks, and document the choice. The identity stays exact either way because the missing week's weight is 0, but the attribution does not. Filling the missing rate with 0 sends an appearing segment's entire w_s1 * r_s1 into the interaction term, since w_s0 = 0 makes both the rate term and the mix term (w_s1 - w_s0) * r_s0 identically zero; a vanishing segment then splits as -w_s0 * r_s0 in rate, -w_s0 * r_s0 in mix and +w_s0 * r_s0 in interaction.
- The convention used below instead imputes the missing week's rate as that week's pooled rate. A vanishing segment then lands wholly in mix at -w_s0 * r_s0, with rate and interaction cancelling; an appearing segment puts w_s1 * r_pooled0 in mix (volume arriving at the average rate) and only w_s1 * (r_s1 - r_pooled0) in interaction (its rate differing from that average). Impute by which week the segment is missing from, never by argument order, or the swap identities below stop holding.
- Guard the division where visitors is 0 so no NaN enters the vectors, because a single NaN poisons every sum. A segment with zero visitors in both weeks contributes exactly 0 and can be dropped; a segment missing from only one week does not contribute 0, and where its contribution lands is settled by the convention above, not by the guard.
- Compute the three components as vectors over segments, then sum. Keep the vectors, because the per-segment contribution table is what turns the decomposition into an explanation.
- Assert that the three components sum to the observed pooled change within floating-point tolerance. This identity is exact, so a mismatch means an implementation bug, not a modelling judgement.
Follow-up
- The mix effect accounts for 0.71 of the 0.84 point drop, driven by paid_social volume. What is your recommendation, and what would change it?
- Why is a two-way split into a counterfactual rate and a residual also exact, and when would you prefer it to the three-way version?
- Segmenting on device and channel leaves a large interaction term. What does that tell you about the choice of segments?
Weekly visit-to-signup conversion split by acquisition channel
From fct_session (session_id, visitor_id, started_at_utc, referrer_channel, is_bot_flagged, consent_state) and fct_event (visitor_id, occurred_at_utc, event_name), compute visit-to-signup conversion for one ISO week, split by channel. session_id is the unique key of fct_session. Denominator: distinct visitor_id with a session starting in the week, is_bot_flagged = FALSE and consent_state <> 'denied'. Numerator: those visitors with a 'signup_completed' event in the same week. Label each visitor with the referrer_channel of their first session in the window. Return channel, visitors, signups and rate, plus one all-channel total row.
Approach
- Build a visitor spine that is one row per visitor: filter sessions to the week, drop is_bot_flagged and consent_state = 'denied', then take the first session per visitor with ROW_NUMBER() OVER (PARTITION BY visitor_id ORDER BY started_at_utc, session_id) = 1 to carry the channel label. Collapsing to one row here is what makes the channel buckets mutually exclusive and the totals additive.
- session_id is the unique key, so that ordering is total and the label is reproducible. If the table carried no unique key you would have to write an explicit tie rule instead, because two sessions on different channels at the identical timestamp would otherwise label the visitor differently between runs.
- Attach the outcome as a semi-join (EXISTS on a signup_completed event for that visitor inside the same week) rather than a join to the event table, so a visitor who fires the event twice does not count twice and inflate the numerator past the denominator.
- Aggregate with COUNT() as visitors and COUNT() FILTER (WHERE signed_up) as signups, and compute the rate as signups::numeric / NULLIF(visitors, 0) so an empty channel returns NULL rather than a division error.
- Produce the total with GROUP BY GROUPING SETS ((channel), ()), which re-sums numerator and denominator for the total row. Averaging the channel rates gives a different and wrong number whenever channel volumes differ, which they always do.
- Verify the spine before trusting the output: COUNT(*) must equal COUNT(DISTINCT visitor_id), and the per-channel visitor counts must sum to the total row.
Worked solution 20 min
- Write the filtered session CTE and check its row count against an unfiltered count, so you know how much volume the bot and consent filters removed.
- Add the ROW_NUMBER first-session pick and assert one row per visitor.
- Add the EXISTS outcome flag and aggregate with FILTER.
- Add GROUPING SETS for the total and format the rate to four decimal places.
- Spot-check one channel by hand: pull its visitor list, count signups directly, compare.
Follow-up
- The denominator is distinct visitors. If a browser release shortens cookie lifetime, what happens to this rate, and how would you tell that apart from a genuine drop?
- A visitor's first session is direct and their signup session is paid search. Your label says direct. When is that the wrong answer for the decision being made?
- How do you roll four weeks into a month, and why is averaging the four weekly rates wrong?
Find reactivation gaps in account paid-period history
fct_subscription_period holds account_id, subscription_id, period_start_utc, period_end_utc, period_status and change_reason. A mid-period plan or seat change closes one row and opens another, so a single continuous paid tenure is often many rows, and an account may hold two overlapping subscriptions. Collapse rows with period_status in ('active','past_due') into continuous tenures per account, treating gaps of three days or less as continuous. Return account_id, tenure_start, tenure_end, and for every tenure after the first, the gap in days that preceded it.
Approach
- Filter to paid rows only: period_status IN ('active','past_due'). Trialing periods are not tenure, and including them turns every trial that never converted into a one-period tenure followed by a fake churn.
- Order by period_start_utc and take a running maximum of all prior ends: MAX(period_end_utc) OVER (PARTITION BY account_id ORDER BY period_start_utc, period_end_utc, subscription_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING). Those three columns are the only stable ordering this schema exposes, so check first that they are unique within an account; if rows tie on all three, the island numbering is order-dependent between runs and you need a real row key before the result is reproducible.
- LAG on its own is wrong here because with overlapping or nested periods the immediately preceding row by start date is not the one that ends latest, so the running maximum is the part that cannot be shortcut.
- Flag a new island when prior_max_end IS NULL OR period_start_utc > prior_max_end + interval '3 days', then number islands with a running SUM of the flag over the same ordering and an explicit ROWS frame.
- Group to (account_id, island) taking MIN(period_start_utc) and MAX(period_end_utc), then LAG(tenure_end) OVER (PARTITION BY account_id ORDER BY tenure_start) to compute the preceding gap in days for every tenure after the first.
- Sanity-check with change_reason, which is the only lineage this schema carries: list its distinct values first, then confirm that rows recording a plan or seat change sit inside a tenure rather than opening one, and that every tenure after the first opens on a row whose reason records a restart rather than an ordinary renewal. Do not reconcile against a churn timestamp on dim_account, which this schema does not define; and where such a column does exist, a cancellation timestamp records when the request was made and routinely sits weeks before the period it ends.
Follow-up
- Why three days of grace? What do 0 and 30 days each do to the count of accounts classed as reactivated?
- An account runs two concurrent subscriptions for different teams. One tenure or two, and what does the revenue reader expect?
- How would you turn these tenures into a monthly gross logo churn series without double-counting an account that churned and returned in the same month?
What is the difference between MLB and MLB Advanced Media?
What is the difference between MLB and MLB Advanced Media?
Approach
- Work from the decision backwards to the evidence you would need.
- Say what you would check first and why it is the highest-information step.
- Clarify what is being asked and what a complete answer would contain.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
What are your favorite statistical functions, and why do they add valu…
What are your favorite statistical functions, and why do they add value to your analysis?
Approach
- Work from the decision backwards to the evidence you would need.
- Say what you would check first and why it is the highest-information step.
- State your assumptions explicitly before working the problem.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Size an activation test before anyone writes code
A team wants to test a new onboarding checklist against the seven-day activation rate: users with is_core_action = TRUE events on at least two distinct UTC dates inside the first seven days, counted from dim_user.account_created_at_utc with is_internal = FALSE. Baseline is 22%. Weekly non-internal signups are 9,000, split evenly between two arms. State the minimum detectable effect after two weeks of enrolment, the enrolment needed to detect a 1.5 percentage point absolute lift, and the earliest date a readout would be honest.
Approach
- Use the two-proportion sizing shortcut n = 16 p(1-p) / d^2 per arm, where d is the absolute lift. The 16 is 2(1.96 + 0.84)^2 = 15.7 rounded, i.e. two-sided alpha 0.05, 80% power, equal arms.
- Invert it for the MDE at fixed n: d = sqrt(16 p(1-p) / n). Two weeks gives 9,000 per arm, so d = sqrt(2.746 / 9000) = 0.0175, which is 1.75 percentage points absolute or 7.9% relative on a 22% base.
- Run it forward for d = 0.015: n = 2.746 / 0.000225 = 12,202 per arm, 24,404 total, about 2.7 weeks of enrolment. Round up to three whole weeks so the enrolment window contains no partial week, since signups are not uniform across weekdays.
- Add the metric's own lag rather than reporting the enrolment end date. Activation is unobservable until seven days after the last enrolled signup, and the definition publishes on an eight-day lag, so the readout lands 29 days after launch.
- Close by testing whether the ask is realistic. If the team's honest prior is a 0.5pp lift, the requirement is about 110,000 per arm and roughly 24 weeks of enrolment, and saying so before launch is the deliverable.
Worked solution 20 min
- Compute p(1-p) = 0.22 x 0.78 = 0.1716 and 16 x 0.1716 = 2.746.
- MDE at two weeks: n = 4,500 per arm per week x 2 = 9,000; d = sqrt(2.746 / 9000) = 0.0175.
- Sample for d = 0.015: n = 2.746 / 0.000225 = 12,202 per arm, which is 2.71 weeks of enrolment, so enrol three whole weeks.
- Readout date = 21 days of enrolment + 8 days of follow-up and publication lag = 29 days from launch.
- State the boundary of what this quarter can answer: below roughly 0.7pp the requirement passes 56,000 per arm and more than a full quarter of enrolment.
Follow-up
- The team also wants trial-to-paid read from the same test. What does a second primary metric do to your alpha, and what would you actually do about it?
- Signups are heavier on weekdays. What goes wrong if enrolment runs 17 days instead of 14?
- The activation definition uses a two-distinct-days threshold. If that were loosened to one day, what happens to the baseline rate and to the sample you need?
Decide whether a one-day core-action drop is real
A daily dashboard counts distinct fct_event.user_id with is_core_action = TRUE, filtered on occurred_at_utc, and is read at 09:00 UTC. This morning it shows yesterday down 22% against the day before. fct_event is partitioned on received_at_utc. You have fct_event, fct_session and dim_user with thirteen months of history. Deliver a one-paragraph verdict, escalate or do not escalate, with the evidence that settles it, before anyone proposes a product hypothesis.
Approach
- Identify which two weekdays the comparison actually spans, then pull the same weekday-pair transition for the last 52 weeks and place the observed 22% inside that distribution. A day-over-day comparison in a product with a weekday pattern is a comparison of two different populations, so the reference class is the same transition historically, not the prior day.
- Measure partition completeness rather than assuming it. For each of the last 30 days compute the share of that day's occurred_at_utc rows that had landed by 09:00 UTC the following morning, split by surface; mobile clients buffer events offline, so the freshest partition is systematically short and the shortfall is not uniform across surfaces.
- Recompute the same series keyed on received_at_utc. If the drop survives on both keys it is not a lateness artefact; if it exists only on occurred_at_utc it is the partition filling in.
- Check the two exclusion flags before segmenting anything: a change in is_bot_flagged coverage or a batch of is_internal accounts entering or leaving moves a distinct-user count with no user behaviour behind it.
- Only if the movement survives all of the above, begin the segment decomposition. Say explicitly in the verdict which of these four checks the movement passed, so the next reader does not repeat them.
Follow-up
- What publication lag would you set for this dashboard, and how would you derive the number rather than pick it?
- If you switch the metric to received_at_utc, what does that break for anyone comparing to historical figures?
- How would you detect the same problem automatically, so a human does not have to notice it each morning?
Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Fix the scope and set a baseline
- Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
- Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
- Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.
Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.
Practice prompt ↗Practice prompt ↗Worked solution ↗02One query pattern, written three times
- Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
- On the third attempt, write the grain of every CTE as a comment before writing its body.
- Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.
Deliverable: Three independent versions of the same query plus a note on what changed between them.
Practice prompt ↗Practice prompt ↗03Only the statistics you will be asked to defend
- Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
- Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
- Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.
Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.
Practice prompt ↗Practice prompt ↗04One case, and the assumptions holding it up
- Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
- Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
- Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.
Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Your own work, timed
- Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
- Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
- Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.
Deliverable: Two timed narratives with one defensible number in the opening line.
Practice prompt ↗Practice prompt ↗06The one full rehearsal, in a longer weekend block
- Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
- Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
- Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.
Deliverable: Mock notes naming three failure moments with a specific fix written under each.
Practice prompt ↗07Taper
- Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
- Re-read only your own notes from this week, and open no new material.
- Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.
Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.
Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.
How would you explain your experience with SAS or R in a project-based…
How would you explain your experience with SAS or R in a project-based context?
Approach
- Quantify the outcome, including what you would not claim credit for.
- Close with what you would do differently, concretely.
- State the situation in two sentences and spend the rest on your reasoning.
Follow-up
- What would you do differently if you ran that project again?
- What did you decide not to do, and why?
How do you handle large, unstructured datasets in a sports-tech enviro…
How do you handle large, unstructured datasets in a sports-tech environment?
Approach
- State the situation in two sentences and spend the rest on your reasoning.
- Quantify the outcome, including what you would not claim credit for.
- Pick a story where you drove the decision, not one where you observed it.
Follow-up
- What would you do differently if you ran that project again?
- How did you know the outcome was caused by your change?
Turn an ambiguous onboarding question into a measurable metric
Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.
Approach
- Recognise what is being probed: whether you convert a goal into a computable predicate without stalling for requirements or guessing in silence. Listing clarifying questions is the generic answer; shipping a defensible default alongside them is the strong one, because the review is in two days and it will happen with or without you.
- Infer the decision behind the request. A question about whether onboarding works, arriving before a planning cycle, usually means whether to staff it next quarter. That points at a rate with visible headroom over several cohorts, not at a descriptive dashboard.
- Write the three questions so that each one changes the SQL. Which population, all signups or only self-serve from dim_user.signup_surface. What counts as working, reaching a core action or completing the onboarding flow_id. Against what bar, last quarter's cohorts or a stated target.
- Propose the default explicitly: seven-day activation on weekly signup cohorts. Numerator, users with is_core_action = TRUE events on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). Denominator, the signup cohort with is_internal = FALSE. Publish with an eight-day lag, and state that the two-distinct-days threshold is a frozen choice rather than a discovery.
- Name the exclusion in the same breath as the number. The series shows whether users activate; it does not establish that onboarding caused the level, which needs a staged rollout or an experiment.
Follow-up
- The director replies that they meant the onboarding flow specifically, not activation. What changes in the query and in the caveats?
- Your cohort metric needs an eight-day lag and the review is in two days. What do you present, and how do you label it?
- Two of your three questions come back unanswered. Which one do you refuse to proceed without?
- 01
How would you explain your experience with SAS or R in a project-based context?
- 02
How do you handle large, unstructured datasets in a sports-tech environment?
- 03
Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.
Is this an official Major League Baseball (MLB) interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Major League Baseball (MLB). Rounds and questions reflect what candidates have reported, not a process Major League Baseball (MLB) has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How long should I expect the hiring process to take?
Timelines vary, but the process generally moves quickly once you are in the interview loop. Ensure you are prepared to move to the next stage shortly after a phone screen.
PracHub interview research ↗Is knowledge of baseball statistics required?
While deep knowledge of baseball analytics is not always a strict requirement for every Data Scientist role, showing an interest in the domain and understanding how MLB operates as a business will significantly differentiate you.
PracHub interview research ↗What is the biggest challenge during the interview?
The biggest challenge is often the intensity of the technical assessment. Stay focused on delivering high-quality, documented work, and ensure you communicate your progress if a deadline feels tight.
PracHub interview research ↗Sources & methodology 3 sources ↗
Official role evidence, timestamped platform data and clearly labeled preparation advice.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
platform · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
platform · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
platform · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
1 digit technology Pvt Data Scientist Interview Questions & Guide 2026
Take-home style practice for data scientists preparing for 1 digit technology Pvt: a messy dataset, a fixed time budget, and a written analysis, reviewed for whether it states its assumptions and whether a reader could reproduce your number
Vericast Data Scientist Interview Questions & Guide 2026
Timed practice for Vericast data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
Verizon Data Scientist Interview Questions & Guide 2026
Product-sense practice for a Verizon data scientist interview: define a product and business analytics metric, pick its guardrails, and argue why a tempting alternative would mislead you.
Vetegrity Data Scientist Interview Questions & Guide 2026
A prep plan for Vetegrity data scientist candidates built on spaced review: write a full answer before opening any solution, keep a running list of the assumptions you failed to state, and repeat every missed exercise three days later inste