As a Data Scientist at A tech company in San Francisco, you are a critical bridge between raw information and strategic innovation. You will be responsible for transforming complex datasets into actionable insights that drive product development, optimize internal processes, and influence high-level decision-making. Your work directly affects how the company's users interact with its platforms and helps keep its technical strategies rooted in empirical evidence.
This role requires a unique blend of technical rigor and business acumen. You will work within cross-functional teams—often collaborating directly with Principal Investigators (PIs), product managers, and engineering leads—to design experiments, build predictive models, and communicate findings to stakeholders at every level. The environment is fast-paced and intellectually demanding, requiring you to remain comfortable with ambiguity while maintaining a high standard of analytical precision.
Because many of the company's teams operate like specialized research labs, you should be prepared to discuss not only your technical proficiency but also your ability to contribute to a collaborative, academic-leaning culture.
Preparation focus
editorialNo round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.
What to demonstrate
- Breadth across SQL, experimentation and product reasoning
- Ability to state assumptions before choosing a method
How to prepare
- Drill the practice exercises below and time yourself
- Prepare three quantified stories about decisions you drove
PracHub editorial advice for the preparation topics above.
Comparing cohort retention curves of different maturities, or building the curve from users who are still present
A cohort four weeks old has no week-8 value, so an average taken across cohorts silently drops young cohorts from the later columns and keeps them in the earlier ones. The curve then bends upward at the tail, and the reading that 'retention is improving over time' is an artefact of which cohorts survived to be measured. The same error appears in the denominator when retention is computed over users active in the current period rather than over the full original cohort, which conditions on survival and guarantees a flattering number. The fix is a triangle: fix the cohort at signup, bound every window on both sides, and only compare cells where every cohort has had the full elapsed time, publishing the rest as blank rather than as a partial average.
Treating last-touch attribution as the causal value of a channel
The attribution label on dim_user is the output of a rule that assigns full credit to whichever touch happened to be recorded last inside a lookback window, and that rule systematically rewards channels that sit close to the conversion, especially branded search and retargeting, which largely intercept demand that already existed. Reallocating spend on those labels moves budget toward the channels that are best at being last, which is why attributed return on ad spend often improves while total signups do not. Nothing in the touchpoint data can settle this, because the counterfactual of not running the channel was never observed. The credible reads are a geo holdout or a scheduled pause, sized in advance on the total-signups metric rather than on the attributed one, and the honest framing in the meantime is that the label describes correlation with conversion and not incremental contribution.
Defining the cohort on a post-treatment condition
Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.
Accepting a metric definition without asking about the denominator
Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Can you explain a complex machine learning project you led and the spe…
Can you explain a complex machine learning project you led and the specific impact it had?
Approach
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Check what information would not exist at prediction time, and exclude it.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- Where could label leakage enter this setup?
- What would you monitor after launch to know the model is still valid?
How would you approach building a model for a dataset with significant…
How would you approach building a model for a dataset with significant missing values?
Approach
- Check what information would not exist at prediction time, and exclude it.
- Set a baseline first, so any model has something honest to beat.
- Say how the offline result would be validated online before it is trusted.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- What would you monitor after launch to know the model is still valid?
Cluster bootstrap for a per-session rate randomised on users
An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.
Approach
- Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
- Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
- Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
- Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
- Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.
Worked solution 35 min
- per_user = df.groupby(['variant','user_id'])['converted'].agg(['sum','size']); split into two arrays per arm.
- point = (t_sum.sum() / t_n.sum()) - (c_sum.sum() / c_n.sum()).
- For b in range(B): idx = rng.integers(0, len(t_sum), len(t_sum)); ratio_t = t_sum[idx].sum() / t_n[idx].sum(); same for control; store the difference. Vectorise by drawing a (B, n) index matrix if memory allows.
- ci = np.percentile(diffs, [2.5, 97.5]); naive_se = sqrt(p_t*(1-p_t)/n_sessions_t + p_c*(1-p_c)/n_sessions_c); naive_ci = point +/- 1.96*naive_se.
- width_ratio = (ci[1]-ci[0]) / (naive_ci[1]-naive_ci[0]).
Follow-up
- Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
- Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
- Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?
Rebuild sessions from raw events with a thirty-minute gap
From fct_event (event_id, visitor_id, occurred_at_utc) alone, rebuild sessions: a new session begins when the gap from that visitor's previous event exceeds 30 minutes, and every session is force-closed at UTC midnight so none spans two calendar dates. Return one row per session with visitor_id, session_start, session_end, event_count and session_date. Do not read fct_session; the point is to reproduce it. Assume duplicate occurred_at_utc values exist for the same visitor.
Approach
- Get the previous timestamp per visitor with LAG(occurred_at_utc) OVER (PARTITION BY visitor_id ORDER BY occurred_at_utc, event_id). The event_id tiebreaker is required, not stylistic: with duplicate timestamps an unstable ordering makes the boundary flags non-deterministic between runs.
- Set a boundary flag when prev IS NULL, or occurred_at_utc - prev > interval '30 minutes', or occurred_at_utc::date <> prev::date. The third disjunct is the midnight rule, expressed as a date change rather than a clock comparison so it holds across any gap length.
- Number the islands with SUM(flag::int) OVER (PARTITION BY visitor_id ORDER BY occurred_at_utc, event_id ROWS UNBOUNDED PRECEDING). Because event_id is unique the ordering is total, so no two rows are peers and RANGE UNBOUNDED PRECEDING would compute exactly the same numbers here. Write ROWS anyway: it is the half of the guard that survives someone later simplifying the ORDER BY back to occurred_at_utc alone, at which point the default RANGE frame gives every row sharing a timestamp one shared running total.
- GROUP BY visitor_id and the island number, then MIN(occurred_at_utc) AS session_start, MAX(...) AS session_end, COUNT(*) AS event_count, session_start::date AS session_date.
- Reconcile against fct_session on one sample day. The counts should agree except for server-emitted events carrying no client session, so a systematic difference beyond those is a bug in the gap rule or in the ordering.
Worked solution 30 min
- Pick one high-volume visitor and dump their ordered event timestamps for a day to trace by hand.
- Add LAG with the tiebreaker and eyeball the computed gaps.
- Add the three-part boundary flag and confirm the first event of each date is flagged.
- Add the running SUM with an explicit ROWS frame, then group and aggregate.
- Compare total event_count against the raw input count, and compare session counts to fct_session for the sample day.
Follow-up
- Why 30 minutes? What does a 5-minute rule do to sessions-per-visitor and to any per-session conversion rate?
- The same person uses phone then laptop. Two visitor_ids, two sessions. What breaks if you sessionise on user_id instead?
- The midnight rule splits an overnight session. Which metrics does that bias, and in which direction?
Read an experiment from first exposure, not assignment
fct_experiment_exposure holds experiment_id, unit_type, unit_id, variant, user_id, assigned_at_utc, first_exposed_at_utc, is_in_analysis_population and planned_end_utc. fct_event holds user_id, occurred_at_utc, is_core_action, and carries events up to a known data cut, :data_cut_utc. For one experiment randomised on unit_type = 'user', return per variant: exposed units, units with at least one core action in the seven days after that unit's own first exposure, the rate, and the variant share of exposed units. Only units whose seven-day window has fully elapsed as of the data cut belong in the readout. Units appearing under more than one variant are excluded from both arms and counted separately.
Approach
- Run the contamination pass as an aggregate, not a window: SELECT unit_id FROM fct_experiment_exposure WHERE experiment_id = :exp GROUP BY unit_id HAVING COUNT(DISTINCT variant) > 1, then anti-join it away. PostgreSQL rejects COUNT(DISTINCT variant) OVER (PARTITION BY unit_id) outright, since DISTINCT is not implemented for window functions; if you want the test inline, MIN(variant) OVER (PARTITION BY unit_id) <> MAX(variant) OVER (PARTITION BY unit_id) is the equivalent that does run.
- Do not resolve contamination by keeping the earliest variant. A unit that saw both arms carries treatment from both, so assigning it to either one biases that arm.
- Define the population as is_in_analysis_population = TRUE AND unit_type = 'user' AND first_exposed_at_utc < planned_end_utc AND first_exposed_at_utc + interval '7 days' <= :data_cut_utc. The horizon filter is what makes the readout reproducible next week instead of drifting with every re-run; the data-cut filter is the one that actually buys seven days of follow-up, since a unit exposed an hour before the horizon otherwise contributes an hour of observation to a seven-day rate.
- Measure the outcome on a per-unit relative window: LEFT JOIN fct_event on user_id with is_core_action = TRUE and occurred_at_utc in [first_exposed_at_utc, first_exposed_at_utc + interval '7 days'). LEFT JOIN so units with no outcome stay in the denominator at zero rather than being deleted by an inner join.
- Check the sample ratio before reading the effect: variant share of exposed units against the intended split, tested as a binomial. Run it on the truncated population as well as on the full exposed set, because if one arm exposes later on average the data-cut filter removes more of that arm and can manufacture a ratio mismatch the randomisation did not have. A mismatch on the full set means the exposure data is not a valid randomisation and invalidates the readout rather than being a footnote under it.
- Report the per-variant rate, the absolute difference, and the fact that the variance unit is unit_id. That is straightforward here only because the grain is already one row per user; a per-session outcome under user randomisation would need a delta-method or bootstrap standard error instead.
Follow-up
- Some units were assigned days before they were exposed. What does analysing the assigned set instead do to the estimated effect, and in which direction?
- The split is 51/49 on 400,000 exposed units. Do you read the result?
- The treatment arm exposes on average two days later than control. What does that do to a fixed calendar outcome window, and which arm does it favour?
How do you determine which metrics to track when evaluating the succes…
How do you determine which metrics to track when evaluating the success of a new feature?
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
What are the differences between supervised and unsupervised learning …
What are the differences between supervised and unsupervised learning in the context of our current research?
Approach
- State your assumptions explicitly before working the problem.
- Clarify what is being asked and what a complete answer would contain.
- Say what you would check first and why it is the highest-information step.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Halve the runtime of a flat test with pre-period data
A test on 28-day core actions per user is powered for six weeks and the team will not wait. You have fct_event (user_id, occurred_at_utc, is_core_action) covering the period before each unit's first_exposed_at_utc in fct_experiment_exposure, and the pre-period count correlates with the in-experiment count at r = 0.6. Apply CUPED, state precisely how much runtime it buys, and handle the units with no pre-period because they signed up after the experiment started.
Approach
- Define the adjusted outcome: Y_cuped = Y - theta (X - Xbar), with theta = Cov(Y, X) / Var(X) estimated on the pooled sample so theta itself carries no treatment effect. It is unbiased because X is pre-exposure and therefore has equal expectation in both arms.
- Quantify the gain exactly: Var(Y_cuped) = Var(Y)(1 - r^2) = Var(Y) x 0.64, so 36% of the variance is removed. Required sample scales with variance, so six weeks becomes 3.8 weeks, and at the original sample the MDE shrinks by sqrt(0.64) = 0.8.
- State the precondition that actually bites: X must be measured strictly before first_exposed_at_utc, not before assigned_at_utc and not before the calendar start of the experiment. Any post-exposure information in the covariate lets the treatment effect leak in and biases the estimate rather than merely failing to help.
- Handle missing pre-periods as a stratum, never by dropping rows. Dropping changes the population; setting X to the pooled mean leaves those units' adjustment at zero, which is unbiased but buys them no reduction, so report the covered share and expect the blended reduction to fall short of 36%.
- Note that stratification is the same mechanism with a categorical covariate. Post-stratifying on device_type and referrer_channel removes the variance those stratum means explain, and it composes with CUPED provided the strata are also fixed before exposure.
Worked solution 30 min
- Build the covariate: for each unit_id in fct_experiment_exposure, count fct_event rows with is_core_action = TRUE and occurred_at_utc in [first_exposed_at_utc - 28 days, first_exposed_at_utc).
- Estimate theta on the pooled sample as Cov(Y, X) / Var(X), and confirm Xbar is statistically indistinguishable between arms before using it.
- Form Y_cuped = Y - theta (X - Xbar) and run exactly the test you would have run on Y.
- For units with no pre-period, set X to the pooled mean so their adjustment is zero, and report what share of the sample that is.
- Report unadjusted and adjusted estimates side by side, with the interval narrower on the adjusted one.
Follow-up
- Here the covariate is the same metric as the outcome. When would you deliberately pick a different one, and what is the risk if that covariate is itself affected by treatment?
- The point estimate moves by 0.4 standard errors when CUPED is applied. Is that reassuring or alarming, and what would you check?
- How does CUPED combine with cluster randomisation on account_id, and at which grain is the covariate built?
Decide whether a one-day core-action drop is real
A daily dashboard counts distinct fct_event.user_id with is_core_action = TRUE, filtered on occurred_at_utc, and is read at 09:00 UTC. This morning it shows yesterday down 22% against the day before. fct_event is partitioned on received_at_utc. You have fct_event, fct_session and dim_user with thirteen months of history. Deliver a one-paragraph verdict, escalate or do not escalate, with the evidence that settles it, before anyone proposes a product hypothesis.
Approach
- Identify which two weekdays the comparison actually spans, then pull the same weekday-pair transition for the last 52 weeks and place the observed 22% inside that distribution. A day-over-day comparison in a product with a weekday pattern is a comparison of two different populations, so the reference class is the same transition historically, not the prior day.
- Measure partition completeness rather than assuming it. For each of the last 30 days compute the share of that day's occurred_at_utc rows that had landed by 09:00 UTC the following morning, split by surface; mobile clients buffer events offline, so the freshest partition is systematically short and the shortfall is not uniform across surfaces.
- Recompute the same series keyed on received_at_utc. If the drop survives on both keys it is not a lateness artefact; if it exists only on occurred_at_utc it is the partition filling in.
- Check the two exclusion flags before segmenting anything: a change in is_bot_flagged coverage or a batch of is_internal accounts entering or leaving moves a distinct-user count with no user behaviour behind it.
- Only if the movement survives all of the above, begin the segment decomposition. Say explicitly in the verdict which of these four checks the movement passed, so the next reader does not repeat them.
Follow-up
- What publication lag would you set for this dashboard, and how would you derive the number rather than pick it?
- If you switch the metric to received_at_utc, what does that break for anyone comparing to historical figures?
- How would you detect the same problem automatically, so a human does not have to notice it each morning?
Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Breadth pass: query fluency
- Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
- For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
- Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.
Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.
Practice prompt ↗Practice prompt ↗Worked solution ↗02Breadth pass: statistics and inference
- Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
- Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
- Rewrite the two weakest answers the following morning from memory in full sentences.
Deliverable: Ten graded answers with an honest count of exact hits.
Practice prompt ↗Practice prompt ↗03Breadth pass: modelling
- Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
- Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
- Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.
Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.
Practice prompt ↗Practice prompt ↗04Breadth pass: product judgement
- Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
- For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
- Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.
Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Depth, first area
- Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
- Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
- Re-solve the two you failed the same evening with notes closed.
Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.
Practice prompt ↗Practice prompt ↗06Depth, second area, and the seam between them
- Repeat the depth protocol on the second-ranked area with the same six-problem structure.
- Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
- Solve your own combined problem end to end and note where the handoff between the two areas cost you time.
Deliverable: One combined problem, solved end to end, with the handoff failure written down.
Practice prompt ↗07Integration and re-measurement
- Re-run the six prompts from day one under the same clock and compare both correctness and time.
- Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
- Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.
Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.
Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.
How do you handle overfitting when working with high-dimensional data?
How do you handle overfitting when working with high-dimensional data?
Approach
- Close with what you would do differently, concretely.
- State the situation in two sentences and spend the rest on your reasoning.
- Quantify the outcome, including what you would not claim credit for.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Explain a wide interval to a non-technical executive
A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.
Approach
- Recognise what is being probed: whether you can carry uncertainty into a decision instead of either hiding it or hiding behind it. The generic answer promises to explain the interval in plain English; the strong one replaces the question 'what is the number' with 'across this range, where does the decision change'.
- Find the threshold before you draft anything. Ask what the pricing case assumes, then compute the conversion drop at which the higher price stops adding revenue: price uplift on the conversions kept against the revenue lost from conversions forgone. That single figure is what makes the range legible.
- Restate the estimate and both bounds in the unit the audience already reasons in. Convert percentage points into monthly first-paid conversions at current trial volume, then into mrr_cents_constant_fx, so the slide reads as money per month rather than as statistics.
- Place the range against the break-even and say which part of it sits on each side. If most of the range clears the threshold, that is a recommendation to proceed with a monitoring plan; if the range straddles it, that is a recommendation to narrow the range first.
- Name what would narrow it and what that costs in weeks, then give one recommendation with an explicit condition for revisiting it. Uncertainty stated without a next step is read as indecision and the midpoint gets used anyway.
Follow-up
- The executive says to give the midpoint and they will manage the risk. What do you do?
- How does the slide change if the interval were a 4.6pp to 0.2pp drop, with no positive outcomes in range?
- Why is a geo holdout the credible read here rather than the attributed channel numbers you already have?
Disagree with a product manager's roadmap claim using data
A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.
Approach
- Recognise what is being probed: whether you can separate a number being right from an inference being wrong, and do it without costing the PM face. The generic answer recites that correlation is not causation; the strong one names the specific confound and proposes the cheapest design that could distinguish the explanations.
- State the alternative concretely. Accounts that connect an integration in week one are accounts that already have a workflow and a technical owner, so week-one intent plausibly drives both the connection and week-four retention. The selection is on intent, which no amount of post-hoc adjustment observes.
- Order the discriminating analyses by cost. First, condition on pre-connection activity by comparing retention within strata of week-one core-action count, which removes the crude version of the confound but not unobserved intent. Second, look for variation in integration availability that was unrelated to intent, such as a staggered release or an outage window. Third, an encouragement design that randomises a prompt to connect and reads the intent-to-treat effect on week-four retention, which is the only version that identifies an effect.
- Run the timing check, because it is nearly free and it is the most persuasive single piece of evidence. If the retention advantage among connectors is already visible before any of them connected, the causal story is largely finished.
- Pre-commit to what would change your mind and say it before you show anything: if the gap survives stratification and the encouragement arm moves week-four retention at all, the feature has a case and you will say so.
- Open the 1:1 by agreeing with the true part, that the correlation is real and worth chasing, then ask what effect size the roadmap plan assumes. That makes the size of the claim the topic instead of its authorship.
Follow-up
- The encouragement test needs six weeks and the roadmap locks in two. What do you recommend in the interim?
- Stratifying on week-one activity closes half the gap. What do you conclude, and what do you still not know?
- How would you word this in the roadmap document so the PM's original number is reframed rather than deleted?
- 01
How do you handle overfitting when working with high-dimensional data?
- 02
A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.
- 03
A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.
Is this an official A tech company in San Francisco interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at A tech company in San Francisco. Rounds and questions reflect what candidates have reported, not a process A tech company in San Francisco has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult is the interview process?
Most candidates describe the process as average in difficulty, but it is rigorous. Focus on being clear in your explanations, and you will find the process manageable.
PracHub interview research ↗Is the take-home assignment really that important?
Yes. It is a primary indicator of your day-to-day work quality. Treat it as a professional deliverable rather than a school assignment.
PracHub interview research ↗What is the company culture like?
Candidates report that the company values collaboration and intellectual honesty. Interviewers are reported to look for people who are excited to learn from others and who can contribute to a supportive, high-achieving environment.
PracHub interview research ↗How long does the process take?
Timelines vary, but you can generally expect the process to span a few weeks from the initial screen to the final decision.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Fetch Rewards Data Scientist Interview Questions & Guide 2026
Product-sense practice for a Fetch Rewards data scientist interview: define a product and business analytics metric, pick its guardrails, and argue why a tempting alternative would mislead you.
1 digit technology Pvt Data Scientist Interview Questions & Guide 2026
Take-home style practice for data scientists preparing for 1 digit technology Pvt: a messy dataset, a fixed time budget, and a written analysis, reviewed for whether it states its assumptions and whether a reader could reproduce your number
xAI Data Scientist Interview Questions & Guide 2026
Communication practice for xAI data scientist candidates: turn an analysis into a recommendation, name the uncertainty honestly, and answer the follow-up a skeptical stakeholder will raise.
Windfall Data Scientist Interview Questions & Guide 2026
Data-quality drills for Windfall data scientist prep: survivorship and selection bias, leakage through time-ordered features, and extreme values where provenance decides the call. A logging fault or unit slip gets corrected or dropped; a ge