As a Data Scientist at Twilio, you sit at the intersection of customer communication data, machine learning infrastructure, and business growth. You drive strategic decisions across product development, growth marketing, and platform engineering by turning massive streams of interaction data into actionable insights. Your core mission is to help hundreds of thousands of businesses and millions of developers craft personalized, reliable customer experiences at global scale.
This role directly influences how Twilio scales its communication APIs, conversational AI agents, and messaging infrastructure. Whether you are building predictive churn models, optimizing engagement funnels, or designing rigorous experimentation frameworks, your work impacts the core revenue-generating engines of the company. You will collaborate closely with product managers, software engineers, and go-to-market teams to shape product roadmaps and define success metrics.
The environment is fast-paced, highly analytical, and deeply embedded in a remote-first culture. You will face complex ambiguity, massive datasets, and the challenge of balancing long-term statistical rigor with rapid product iteration. Success here requires a blend of advanced technical capability, strong product intuition, and the ability to translate complex statistical concepts into clear business recommendations.
Recruiter Screen
reportedData Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.
What to demonstrate
- Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
- Whether you name what you have not done instead of stretching to cover every line of the posting
- Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage
How to prepare
- Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
- Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
- Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live
Hiring Manager Screen
reportedMuch of this round runs on your own history, but the manager is not collecting a project list. They are working out what it is like when something goes wrong on your watch: how late the bad news tends to arrive, and whether a number you hand over has been checked by anyone including you. That is why the strongest material is a project where you can describe the part that did not work and what it cost. A result you cannot take full responsibility for, however clean, gives them nothing to trust you with afterwards.
What to demonstrate
- Whether you volunteer the limits of a result you are proud of, or wait to be pushed onto them
- How errors surfaced in your past work, and whether you or somebody else found them
- Whether the scope you claim matches the level of detail you can still produce about it
- What you did the first time a stakeholder acted on something of yours that turned out to be wrong
How to prepare
- Rebuild one headline figure from memory down to the join and the filter, so a question about the denominator does not stall the conversation
- For each project you raise, write the sentence you would say to someone who had already acted on a number that later turned out wrong
- Mark which parts of a project were yours and which belonged to other people, and state that boundary yourself before anyone asks
Take-Home Technical Challenge
reportedA take-home is graded as an argument, not as a notebook. Somebody reads the submission without you in the room, so every choice has to survive on the page: why the question was framed this way, and what was deliberately left out. The gap between a strong and a weak submission is almost never model quality. It is whether the writeup names the specific question it answers and commits to a recommendation, including what evidence would overturn it. A high-accuracy model attached to no conclusion reads as effort that stopped before the decision.
What to demonstrate
- Whether the question you answered is stated outright, and whether it is the question the prompt posed rather than an easier neighbour of it
- Whether the recommendation is specific enough to act on, with the uncertainty attached to it instead of parked in a caveats section at the end
- Whether analytical choices such as the metric definition, the population filter and the time window are justified in the prose, not merely visible in code
How to prepare
- Take a dataset you have already worked with, write the one-paragraph conclusion first, then check whether the analysis you were planning actually supports it and cut whatever does not
- Practise stating a metric in one sentence that fixes the population, the time window and the denominator, then confirm your query computes exactly that sentence and nothing adjacent to it
- Hand a draft to someone outside the problem and ask them to tell you back what you recommended and why; anything they cannot recover is not on the page yet
Technical Loop
reportedWhere a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.
What to demonstrate
- Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
- Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
- Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
- Whether you ask what decision is actually on the table before explaining anything
How to prepare
- Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
- For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
- Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else
Final Loop & Bar Raiser
reportedA loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.
What to demonstrate
- Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
- Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
- Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience
How to prepare
- Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
- For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
- Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story
PracHub editorial advice for the preparation topics above.
Reading consumption metrics before the metering lag window has closed
Usage pipelines land late and correct themselves, which is exactly what is_restated and restated_at record. A dashboard queried on day T sees a partially populated tail for the last several days, so the most recent points always slope downward and always look like a regression. Analysts then explain the artefact, and sometimes ship a change to fix it. Establish the empirical settling time by measuring how much a given usage_date's total moves between first_written_at and its final value, exclude that many trailing days from every reportable figure, and never compare a fresh period against a settled one.
Reporting a mean over accounts when account revenue is heavy-tailed
When a small number of accounts hold most of the revenue, the sample mean is dominated by whichever of them happens to be in the sample, and the sample variance keeps growing as more data arrives instead of stabilising. In that regime the usual central-limit-based confidence interval understates uncertainty, and a single renewal or a single large account's batch job can flip the sign of a measured effect. The fixes are to pre-register a winsorisation or capping rule before looking at the outcome, to report account counts crossing a threshold alongside the revenue figure, or to define the estimand on a bounded transform. Choosing the cap after seeing the result is a separate and worse problem, because the cap then encodes the answer.
Reading a dozen metrics with no multiplicity control
Nominate one primary metric before launch and treat the rest as guardrails or exploratory, with Bonferroni or Benjamini-Hochberg applied when you intend to make claims from them. Twenty independent tests at 0.05 under the null produce at least one false positive about 64 percent of the time.
Never asking what decision the analysis will inform
Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
How do you test for heterogeneous treatment effects across different c…
How do you test for heterogeneous treatment effects across different customer segments in an observational study?
Approach
- Write down the assumption the method needs before you use the method.
- Quantify uncertainty explicitly rather than reporting a point estimate alone.
- Say what the estimate is of, and over what population it generalises.
Follow-up
- Which assumption here is most likely to be violated in practice?
- What sample size would you need to detect an effect half this size?
How would you approach building a churn prediction model using histori…
How would you approach building a churn prediction model using historical customer usage logs and support ticket data?
Approach
- Say how the offline result would be validated online before it is trusted.
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
Follow-up
- Where could label leakage enter this setup?
- What would you monitor after launch to know the model is still valid?
Sessionise an API event stream with a 30-minute inactivity gap
fct_api_request arrives as a DataFrame with account_id, user_id, request_at (tz-aware UTC), traffic_class and http_status, roughly 5 million rows. Assign a session_id to every human-attributable request: drop rows where user_id is null or traffic_class is in ('ci','synthetic_monitor','load_test'), then open a new session whenever the gap since that user's previous remaining request exceeds 30 minutes. Return the filtered frame plus session_id, and a per-session summary with user_id, account_id, session start, session end and request count. Do not loop over rows.
Approach
- Settle the filter-then-gap ordering before writing code. Removing CI and synthetic rows changes the gaps, so sessionising the raw stream and filtering afterwards is a different answer; the definition given filters first, and the two diverge most for accounts whose CI runs every ten minutes.
- Sort once by (user_id, request_at) with a stable kind, then gap = df.groupby('user_id', sort=False).request_at.diff(). The first row of each user yields NaT, which is exactly the boundary condition you want rather than a special case to patch.
- new_session = gap.isna() | (gap > Timedelta(minutes=30)); session_id = new_session.cumsum(). The cumsum runs over the whole sorted frame and therefore produces globally unique ids in one pass; a per-user cumcount collides across users and forces a composite key on every downstream join.
- Build the summary with a single groupby('session_id').agg(...). user_id and account_id can be carried with 'first' only because the sort key groups them — state that dependency, since it silently breaks if someone later re-sorts the frame.
- Decide explicitly what a session means when one user_id holds memberships in several accounts: either add account_id to the sort and group keys, or document that sessions may cross accounts. Leaving it undecided produces sessions whose account_id is whichever row sorted first.
Worked solution 30 min
- human = df[df.user_id.notna() & ~df.traffic_class.isin(['ci','synthetic_monitor','load_test'])].sort_values(['user_id','request_at'], kind='mergesort').reset_index(drop=True)
- gap = human.groupby('user_id', sort=False).request_at.diff(); human['session_id'] = (gap.isna() | (gap > pd.Timedelta(minutes=30))).cumsum()
- summary = human.groupby('session_id').agg(user_id=('user_id','first'), account_id=('account_id','first'), start=('request_at','min'), end=('request_at','max'), n_requests=('request_at','size')).reset_index()
- assert summary.n_requests.sum() == len(human) and human.groupby('session_id').user_id.nunique().max() == 1
Follow-up
- Where does 30 minutes come from, and how would you pick it from this data instead of from convention?
- An engineer reused their personal key for a nightly batch job, so machine traffic carries a human user_id. How would you detect that, and should those requests form sessions?
- How much does the session count change if you sessionise before dropping CI traffic rather than after?
How would you optimize a slow-running aggregation query joining multip…
How would you optimize a slow-running aggregation query joining multiple multi-terabyte event tables?
Approach
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- State the window function and its partition and ordering out loud before writing it.
Follow-up
- How does the query change if the join becomes one-to-many?
- How would you verify this result without re-running the same query?
Extract month-over-month retention cohorts using advanced SQL joins an…
Extract month-over-month retention cohorts using advanced SQL joins and conditional aggregation.
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Say which table is the grain you start from, and join outward from it.
- State the window function and its partition and ordering out loud before writing it.
Follow-up
- How does the query change if the join becomes one-to-many?
- How would you verify this result without re-running the same query?
Sessionise interactive API traffic with a thirty-minute inactivity gap
fct_api_request carries account_id, user_id (null for service accounts), request_at, traffic_class and http_status. Using only traffic_class = 'interactive' rows with a non-null user_id, group each seat's requests into sessions on a 30-minute inactivity threshold: a request more than 30 minutes after the previous request from the same (account_id, user_id) opens a new session. For one ISO week return, per account, the session count, the median session duration in minutes and the median requests per session. A single-request session has a duration of zero.
Approach
- Filter first: traffic_class = 'interactive' and user_id IS NOT NULL. Machine traffic has no sessions in any useful sense, and leaving CI or batch rows in produces sessions that are really cron schedules.
- Get the previous timestamp with LAG(request_at) OVER (PARTITION BY account_id, user_id ORDER BY request_at). Partitioning by user_id alone stitches one person's work across two different accounts into one fabricated session, because a human holds memberships in several accounts.
- Flag a boundary where the lag is NULL or request_at - lag > interval '30 minutes'. Decide and state whether exactly 30 minutes continues the session; strictly greater is the conventional choice and needs to be written down either way.
- Assign session ids with SUM(boundary::int) OVER (PARTITION BY account_id, user_id ORDER BY request_at ROWS UNBOUNDED PRECEDING), the standard running-count construction for islands.
- Roll up to sessions with min(request_at), max(request_at) and count(*), then to accounts with percentile_cont(0.5) WITHIN GROUP (ORDER BY ...). Use medians, not means: session length is strongly right-skewed and one long-running client dominates the average.
Worked solution 30 min
- Build the filtered week: interactive rows with user_id IS NOT NULL inside [week_start, week_start + 7 days).
- Add prev_at via LAG(request_at) OVER (PARTITION BY account_id, user_id ORDER BY request_at) and derive is_new_session = (prev_at IS NULL OR request_at - prev_at > interval '30 minutes').
- Add session_seq = SUM(is_new_session::int) OVER (PARTITION BY account_id, user_id ORDER BY request_at ROWS UNBOUNDED PRECEDING).
- Aggregate to sessions by (account_id, user_id, session_seq) taking min, max and count, with duration_minutes = EXTRACT(EPOCH FROM max - min) / 60.
- Aggregate to accounts with count(*) AS sessions, percentile_cont(0.5) WITHIN GROUP (ORDER BY duration_minutes) and percentile_cont(0.5) WITHIN GROUP (ORDER BY request_count).
Follow-up
- Sessions belonging to colleagues in one account are correlated. What does that do to a t-test on session length across an experiment arm?
- A long-poll or streaming endpoint keeps a connection open for hours. How do you stop it reading as one twelve-hour session?
How do you balance competing product metrics, such as message delivery…
How do you balance competing product metrics, such as message delivery speed versus system cost and efficiency?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
A key engagement metric dropped by fifteen percent week-over-week. Wal…
A key engagement metric dropped by fifteen percent week-over-week. Walk through your framework to diagnose the root cause.
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
How would you design a metric to evaluate the long-term health and sti…
How would you design a metric to evaluate the long-term health and stickiness of a developer-facing communication API?
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
How would you define and measure the success of a new conversational A…
How would you define and measure the success of a new conversational AI feature on the Twilio platform?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
How do you determine the required sample size and minimum detectable e…
How do you determine the required sample size and minimum detectable effect for an experiment with high baseline variance?
Approach
- State the primary metric and the minimum effect worth shipping, then size the test.
- Decide the analysis before seeing data, including how long it runs and when you look.
- Say whether units interfere with each other, and switch design if they do.
Follow-up
- What would you conclude if the result is positive but the test is underpowered?
- How would you handle interference between treated and control units?
Explain how you establish statistical significance when dealing with h…
Explain how you establish statistical significance when dealing with highly skewed revenue metrics in an online experiment.
Approach
- Name the randomisation unit first; it decides the variance and what the test can detect.
- Say whether units interfere with each other, and switch design if they do.
- Decide the analysis before seeing data, including how long it runs and when you look.
Follow-up
- What would you do if you could not randomise at all?
- What would you conclude if the result is positive but the test is underpowered?
Suppose we want to launch a feature that recommends optimal messaging …
Suppose we want to launch a feature that recommends optimal messaging channels to developers. How would you structure the product evaluation framework?
Approach
- Say what you would check first and why it is the highest-information step.
- Clarify what is being asked and what a complete answer would contain.
- State your assumptions explicitly before working the problem.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Estimate a staggered overage-price change without a control arm
On 2026-03-01 the overage rate for sku_code = 'ingest_gb' rose 18% for accounts whose dim_account.billing_country falls in six countries. Each account moves to the new rate at its own renewal, so treatment switches on across twelve months. Randomisation was never available. Using fct_usage_daily and fct_subscription_period, estimate the effect on net_amount_cents per account and on gross logo retention. Specify the estimator, the identifying assumption and how you test it, how you handle accounts that churn out of the panel, and how you do inference with six treated countries.
Approach
- Fix the estimand before the estimator. Effect on revenue per account and effect on retention are different questions, and the second determines whether the first is worth having: an 18% rate rise that raises revenue while costing renewals is a loss. Define both on a cohort frozen before 2026-03-01, carrying churned accounts at zero revenue rather than dropping them, so attrition cannot masquerade as a revenue effect.
- Do not run a single two-way fixed-effects regression. With treatment switching on at different renewal dates and effects that vary with time since treatment, the two-way fixed-effects coefficient is a variance-weighted average that includes comparisons of later-treated accounts against already-treated ones, and those comparisons can carry negative weights. Use a cohort-by-cohort estimator such as Callaway-Sant'Anna, or a stacked event study using only not-yet-treated controls, and report the event-study path rather than one number.
- Treat parallel trends as a maintained assumption to be probed, not a result to be shown. Plot pre-treatment lead coefficients with intervals and state that flat leads are consistent with the assumption rather than proof of it. Handle anticipation explicitly: accounts notified of the increase before their renewal can pull ingest volume forward, so exclude a pre-renewal anticipation window and check whether the leads move when you do.
- When the six treated countries are dominated by a few large accounts, the group mean is not a stable object and difference-in-differences on it will be noisy in a way its standard error will not reflect. Build a synthetic control at the country level from a donor pool of untreated countries, weighted to match the pre-period trajectory of net revenue per account, and report the placebo distribution across donor countries instead of a conventional standard error.
- Do inference at the level treatment was assigned, which is the country. Six treated clusters is far too few for cluster-robust standard errors, which are badly downward-biased below roughly forty clusters. Use a wild cluster bootstrap, and report a randomisation-inference p-value from placebo assignments of treated status alongside it.
- Apply the domain's data hygiene or the whole estimate is contaminated: exclude is_internal accounts, exclude the trailing metering settling window so the final months are not artificially low, and join as-of to the fct_subscription_period version live on each date rather than to is_current, which would price last year's usage at this year's contract.
Worked solution 45 min
- Build a balanced monthly account panel from fct_usage_daily joined as-of to fct_subscription_period, excluding is_internal accounts and carrying churned accounts forward at zero revenue.
- Define treatment cohorts by renewal month, estimate group-time average treatment effects, and aggregate to an event-study path with leads from -6 to -1 and lags from 0 to +11.
- Fit a country-level synthetic control on pre-period net revenue per account and generate the placebo distribution by refitting on each donor country in turn.
- Compute a wild cluster bootstrap p-value clustered on country and report the randomisation-inference p-value from the placebo distribution beside it.
- Repeat the full path for gross logo retention, restricted each month to the renewal-eligible base plus the 45-day grace window.
Follow-up
- Your event study shows a significant lead coefficient two months before renewal. Is that anticipation, a violation of parallel trends, or a coding error, and what distinguishes them?
- Revenue per account rose 6% and gross logo retention fell 1.8 points. How do you combine those into a single recommendation, and over what horizon?
- One treated country contains an account holding 20% of that country's revenue. What does that do to the synthetic control fit, and what would you do about it?
Net revenue retention jumps sixteen points in one month
Trailing-twelve-month net revenue retention printed around 108 percent for months and now reads 124 percent, with no unusual deals closed. The query sums arr_cents from fct_subscription_period (account_id, arr_cents, term_start_date, term_end_date, amendment_type, superseded_by_id, is_current, booked_at) filtered on is_current = true at month M, across accounts holding arr_cents > 0 at month M-12. Find the defect, correct the number, and rewrite the definition so the next person cannot reintroduce it.
Approach
- Audit the grain before the arithmetic: count account_ids holding more than one row with is_current = true and superseded_by_id null. A versioned contract table that double counts one amendment batch inflates the numerator while leaving the denominator untouched.
- Replace is_current with an as-of selection on both dates, taking the version whose term_start_date and term_end_date bracket the reporting date and tie-breaking on latest booked_at. The numerator is read as of M and the denominator as of M-12; neither uses today's live version.
- Verify the cohort is frozen. The account set is fixed at M-12 and nothing acquired since may enter the numerator, so check that no join to a current-period table quietly re-admits new accounts.
- Confirm the estimand is a ratio of sums rather than a mean of per-account ratios. Contraction is floored at zero while expansion is unbounded, so the two constructions differ systematically and the second is far noisier.
- Reissue the definition with the failure modes written into it: exactly one row per account per date by construction, cohort frozen at M-12, churned accounts contributing zero rather than dropping out of the numerator.
Follow-up
- A churned account should contribute zero rather than disappear. What does the ratio do under each treatment, and which one is correct?
- How would you unit-test this metric so a future amendment batch with the same defect fails a check instead of reaching a board slide?
Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Fix the scope and set a baseline
- Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
- Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
- Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.
Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02One query pattern, written three times
- Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
- On the third attempt, write the grain of every CTE as a comment before writing its body.
- Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.
Deliverable: Three independent versions of the same query plus a note on what changed between them.
Practice prompt ↗Practice prompt ↗Practice prompt ↗03Only the statistics you will be asked to defend
- Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
- Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
- Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.
Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.
Practice prompt ↗Practice prompt ↗Practice prompt ↗04One case, and the assumptions holding it up
- Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
- Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
- Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.
Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗05Your own work, timed
- Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
- Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
- Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.
Deliverable: Two timed narratives with one defensible number in the opening line.
Practice prompt ↗Practice prompt ↗06The one full rehearsal, in a longer weekend block
- Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
- Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
- Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.
Deliverable: Mock notes naming three failure moments with a specific fix written under each.
Practice prompt ↗Practice prompt ↗07Taper
- Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
- Re-read only your own notes from this week, and open no new material.
- Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.
Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.
Tell me about a time when you had to push back on a stakeholder's prod…
Tell me about a time when you had to push back on a stakeholder's product request using data-driven insights.
Approach
- Name the disagreement or constraint, and how you resolved it with evidence.
- Close with what you would do differently, concretely.
- State the situation in two sentences and spend the rest on your reasoning.
Follow-up
- What would you do differently if you ran that project again?
- How did you know the outcome was caused by your change?
Defend a churn number twelve times the one in the board deck
You recompute logo churn on the renewal-eligible base from fct_subscription_period, counting only accounts whose term_end_date fell in the month and allowing a 45-day grace for late paperwork. Annualised, about 14 percent of accounts that reach a renewal date do not renew. A revenue leader has been quoting 1.2 percent to the board for two quarters, computed by dividing non-renewed accounts by the entire customer base in each month and printing that monthly figure with no period attached. You have 20 minutes with that leader and the finance lead. Decide which number is reported from now on, and what happens to the two quarters already published.
Approach
- The interviewer is probing whether you can hold a correct definition under social pressure without turning it into a competence dispute. Open by reproducing their 1.2 percent exactly, with their denominator and their months, so the disagreement is arithmetic both sides can see rather than a claim about who was careless.
- Separate the two defects, because they are different in kind. The denominator is wrong: on annual contracts only about one twelfth of the base reaches a renewal date in any month, so an account eleven months from renewal sits in the denominator while being structurally incapable of entering the numerator, which suppresses the rate by a factor near twelve. The period is merely unstated: a monthly figure printed beside annual revenue targets gets read as an annual rate.
- Say out loud that those two defects nearly cancel in the level, before the leader finds it. Twelve times 1.2 percent is about 14 percent, which is your number. That is the strongest thing you can say in the room, because it proves both figures rest on the same non-renewal count and moves the meeting onto which denominator and which period get published rather than onto whose query is right.
- The level is recoverable; the series is not. Non-renewals in a month are the eligible base for that month times the churn rate, so dividing by a fixed whole base makes the published line proportional to how many contracts happen to come up that month. Where signings cluster at quarter ends, the eligible base in a quarter-end month can be several times a quiet month's, and the month-over-month moves the board has been reading as satisfaction are the signing calendar.
- Separate the measurement change from a business change. Nothing got worse this week; the loss rate was always this. Bring net revenue retention over the same period as a ratio of sums on a cohort frozen twelve months earlier, because logo churn concentrated in small accounts can sit beside healthy revenue retention, and that combination is the actual story.
- Offer a migration path rather than a correction. Report both rates for one quarter with a written bridge, restate the prior two quarters in an appendix instead of silently, and pin the definition, including the period it is stated over, somewhere finance and product both read it. Concede the limits of your own number: the 45-day grace means the most recent 45 days are not reportable, and churn must be dated on term_end_date rather than on updated_at. A strong answer volunteers this; a generic one only defends.
Follow-up
- The leader multiplies their monthly figure by twelve, lands on your annual number, and concludes nothing was ever wrong. What do you say?
- The leader says publishing the corrected rate costs the team its credibility with the board this quarter. What do you do?
- Gross logo retention worsened while net revenue retention improved. Which do you lead with, and what does the combination tell you about who is leaving?
Scope an open-ended request to predict account churn
A customer success director asks for a list of accounts about to churn. You know only that the team has six people and that contracts are annual. Available data is fct_subscription_period, fct_usage_daily, fct_api_request, fct_support_ticket and dim_account. Before writing any code, produce the questions you need answered, a proposed definition of about to churn, and the shape of the artefact you would hand back, including the operating point that turns a score into a decision.
Approach
- The interviewer is probing whether you convert a vague request into a decision with a capacity constraint attached. A candidate who starts talking about model families has already failed the exercise.
- Pin the event and the horizon first. Churn is only possible at term_end_date, so the population is accounts renewing in the next 60 to 90 days, not the whole base. Ask explicitly whether contraction and downgrade count as churn or only full non-renewal, because the three have different base rates and different interventions.
- Pin the action and the capacity. Six people times a realistic number of meaningful interventions per week gives k, and k is what the list is ranked to. Evaluate on precision at k rather than a global AUC over accounts that will never be contacted.
- Audit leakage before choosing features. Every feature needs a timestamp proving it existed before the prediction date. A downgrade amendment, a churn reason code, and a ticket opened after the renewal conversation started are all leaks that will make the offline number look excellent and the live list useless.
- Ask for the counterfactual now rather than later. Coverage is assigned deliberately, so without a held-out slice agreed at the start the intervention can never be evaluated, and you will be asked for its impact in nine months regardless.
- Propose the smallest artefact that closes the loop: a weekly ranked list sized to capacity with two or three inspectable reasons per row, plus a stated policy for accounts below the line.
Follow-up
- The director insists all accounts are in scope, not only those renewing soon. How do you answer without simply refusing?
- Historical non-renewals number about 30 a year. At what point do you tell them a model is the wrong tool and a rules list is better?
- Which candidate features would you drop purely because you cannot date them?
- 01
Tell me about a time when you had to push back on a stakeholder's product request using data-driven insights.
- 02
You recompute logo churn on the renewal-eligible base from fct_subscription_period, counting only accounts whose term_end_date fell in the month and allowing a 45-day grace for late paperwork. Annualised, about 14 percent of accounts that reach a renewal date do not renew. A revenue leader has been quoting 1.2 percent to the board for two quarters, computed by dividing non-renewed accounts by the entire customer base in each month and printing that monthly figure with no period attached. You have 20 minutes with that leader and the finance lead. Decide which number is reported from now on, and what happens to the two quarters already published.
- 03
A customer success director asks for a list of accounts about to churn. You know only that the team has six people and that contracts are annual. Available data is fct_subscription_period, fct_usage_daily, fct_api_request, fct_support_ticket and dim_account. Before writing any code, produce the questions you need answered, a proposed definition of about to churn, and the shape of the artefact you would hand back, including the operating point that turns a score into a decision.
Is this an official Twilio interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Twilio. Rounds and questions reflect what candidates have reported, not a process Twilio has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult is the interview process, and how much preparation time should I plan for?
The interview loop is rigorous and places heavy emphasis on core fundamentals like A/B testing, SQL proficiency, and product sense. Most candidates benefit from four to six weeks of dedicated preparation, focusing heavily on practicing live coding and structuring open-ended product metrics cases.
PracHub interview research ↗What differentiates successful candidates from those who do not pass?
Successful candidates stand out by structuring their answers logically, grounding their product recommendations in clear metric frameworks, and communicating their assumptions proactively. They do not just jump into calculations; they first clarify the business objective and consider potential edge cases.
PracHub interview research ↗What is the company culture like for data scientists at Twilio?
The engineering and data culture is remote-first, autonomous, and highly collaborative. Data scientists are treated as strategic partners rather than service resources, meaning you will have genuine ownership over the analytical direction of your product area.
PracHub interview research ↗How long does the typical interview process take from start to finish?
From the initial recruiter contact to the final decision, the typical loop spans approximately three to four weeks, depending on scheduling coordination and team availability.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
[24]7.ai Data Scientist Interview Questions & Guide 2026
Modeling drills for [24]7.ai data scientist prep: choosing a baseline, reading regression coefficients honestly, handling class imbalance, and defending a model you would actually ship.
Xcellent Technology Solutions Data Scientist Interview Questions & Guide 2026
Modeling drills for Xcellent Technology Solutions data scientist prep: choosing a baseline, reading regression coefficients honestly, handling class imbalance, and defending a model you would actually ship.
Wolters Kluwer Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for Wolters Kluwer data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.
Zscaler Data Scientist Interview Questions & Guide 2026
Data-quality drills for Zscaler data scientist prep: survivorship and selection bias, leakage through time-ordered features, and extreme values where provenance decides the call. A logging fault or unit slip gets corrected or dropped; a gen