The role of a Data Scientist at Protagonist is critical in shaping the company's data-driven strategies and enhancing product offerings. As a Data Scientist, you will leverage advanced analytics and machine learning techniques to derive insights from vast datasets, ultimately driving decisions that affect users and the business. Your work will have a direct impact on optimizing processes, enhancing user experiences, and informing product development across various teams, making it an integral part of Protagonist's mission.
At Protagonist, you will engage in complex problem-solving tasks within a collaborative environment, contributing to innovative products that align with customer needs and market demands. The role is not only about data analysis; it encompasses strategic influence, requiring you to communicate findings effectively to stakeholders and help guide the company’s direction based on quantitative insights. This dynamic field offers ample opportunity for professional growth, as you will continually encounter new challenges in a rapidly evolving landscape.
Initial Screening
reportedMost candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.
What to demonstrate
- Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
- Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
- The substance of the questions you ask back, which an experienced screener reads as a level signal
How to prepare
- Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
- Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
- Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered
Technical Assessment
reportedBefore anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.
What to demonstrate
- Whether an ambiguous term becomes a specific column and filter before any computation happens
- Whether you read the schema for keys and cardinality rather than only for column names
- Whether the result answers the question at the grain it was asked at, per user or per session or per day
How to prepare
- Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
- On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
- Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over
Behavioral Interview
reportedBehavioural answers from data candidates get audited in a way that answers from other roles do not. When you say a model lifted retention, the next question is the denominator, the window, and how you knew the lift was not seasonal. So attach the measurement to each claim while you tell it: what the metric was before, over what period, and against what comparison. Numbers with no baseline read as rounded-up memory, and one unsupported figure tends to make the rest of the story sound rehearsed.
What to demonstrate
- Whether each impact number arrives with a baseline, a window and a comparison, or as a bare percentage
- Whether you can name the method that attributed the effect to your work (an experiment, a staged rollout, a seasonal control) or concede the link was correlational
- Whether the magnitudes stay internally consistent when the interviewer multiplies them against the scale you described earlier
How to prepare
- For each story, write the impact line as metric, value before, value after, window, and how attribution was established. Any line missing two of those five is a follow-up you will answer badly.
- Re-derive one headline number from the source table rather than the deck that reported it. Resume numbers drift upward across retellings.
- Decide in advance which figures you cannot share, and prepare the ratio or relative change you can give instead, so a confidentiality limit does not read as evasion.
Final Interviews
reportedA day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.
What to demonstrate
- Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
- Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
- Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method
How to prepare
- Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
- Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
- Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
- Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times
PracHub editorial advice for the preparation topics above.
Computing monthly churn against the entire customer base when contracts are annual
An annual contract has no opportunity to churn except at its renewal date, so an account that is eleven months from renewal is in the denominator while being incapable of appearing in the numerator. The resulting rate is smaller than the real one by roughly the ratio of the base to the renewal-eligible base, and it oscillates with the seasonality of when deals were originally signed rather than with anything about the customers. The corresponding trap on the other side is counting a churn on the date the record was updated rather than on term_end_date, which shifts losses into whichever month the operations team did its paperwork.
Randomising an experiment at the user level when users share an account
Two problems fire at once. Colleagues in one workspace see each other's work and talk to each other, so a treated user changes the behaviour of a control user in the same account, which violates the no-interference assumption and biases the estimate toward zero. Separately, outcomes within an account are strongly correlated, so the effective sample size is roughly n / (1 + (m - 1) * rho) for m users per account and intra-class correlation rho, not n. With rho around 0.3 and twenty users per account that is a design effect near 6.7, meaning a user-level confidence interval is about two and a half times narrower than it should be and results cross significance thresholds on noise alone. Randomise the account and cluster the standard errors.
Dropping rows with missing values without naming the mechanism
Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.
Stopping an experiment the moment it crosses significance
Fix the sample size or duration before launch, or use a method built for continuous monitoring such as a sequential test, always-valid confidence intervals, or group-sequential boundaries. Repeatedly checking a fixed-horizon p-value against 0.05 pushes the real false-positive rate well above 5 percent.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Describe your experience with machine learning algorithms. Which ones …
Describe your experience with machine learning algorithms. Which ones have you implemented?
Approach
- Check what information would not exist at prediction time, and exclude it.
- Say how the offline result would be validated online before it is trusted.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- Where could label leakage enter this setup?
- How would you choose the decision threshold, and who owns that choice?
What steps would you take to improve the performance of an underperfor…
What steps would you take to improve the performance of an underperforming model?
Approach
- Set a baseline first, so any model has something honest to beat.
- Check what information would not exist at prediction time, and exclude it.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- Where could label leakage enter this setup?
- How would you choose the decision threshold, and who owns that choice?
Permutation-test a consumption experiment randomised at account level
An experiment randomised 900 accounts into two arms. You have one row per account: account_id, arm, consumption_28d (billable units after launch) and consumption_pre (the 28 days before). Consumption is heavy-tailed and the largest account is several percent of the total. Write a permutation test from scratch: winsorise at the pooled 99th percentile as a pre-registered rule, use the difference in arm means of the winsorised outcome as the statistic, and obtain a two-sided p-value from 20,000 relabellings of the account-level arm vector. Report the observed effect, the p-value, and the same test on a CUPED-adjusted outcome.
Approach
- Be precise about what the permutation test needs. Under the sharp null of no effect for any account, the outcomes are exchangeable across arm labels, and the test is valid for ANY statistic T(outcomes, labels) provided the identical function is applied to the observed labels and to all 20,000 relabellings. The pooled 99th percentile is a function of the outcome vector alone, so recomputing it inside the loop returns the same number 20,000 times: that is wasted CPU, not a bias, and hoisting it out is an optimisation rather than a correctness fix. Say plainly that capping at all changes the estimand from mean consumption to mean capped consumption; it is not a neutral cleaning step.
- The mistake that does invalidate the test is an asymmetry between the observed statistic and the permuted ones, and the easiest way to create it is to derive the cleaning rule from the observed arm labels and then freeze it — winsorise each arm at its own observed 99th percentile, hold those two caps fixed, and permute. The observed value is then computed with caps matched to its own partition while every relabelling is scored with caps belonging to a different one, so the null distribution no longer answers the question the p-value claims to answer. A per-arm cap recomputed consistently inside every permutation is a valid test, but it estimates a contrast whose two sides are capped at different thresholds, so prefer the pooled cap on estimand grounds and pre-register it.
- Permute the account-level arm vector, because the account is the randomisation unit. Relabelling anything finer — users, workspaces, requests — generates a null distribution narrower than the design actually supports and returns p-values that are anti-conservative.
- Vectorise the null: tile the treatment indicator into a (B, n) matrix and permute along axis 1 with rng.permuted(..., out=...). The statistic is a difference of means, so the treated sum alone determines it and the whole null is one matrix-vector product. Use the two-sided p-value (1 + count(|stat_perm| >= |stat_obs|)) / (B + 1); the plus-one on each side is not cosmetic, it keeps the p-value away from exactly zero and keeps the test valid at finite B.
- For CUPED, fit theta = cov(y, x) / var(x) on the pooled data and use that same theta for the observed statistic and every relabelling. Pooled theta, like the pooled cap, carries no label information, so where in the loop you compute it is again only a performance question; fitting theta within arms is what goes wrong, because the adjusted outcome then depends on the labels and an observed-label fit frozen across all 20,000 relabellings breaks the match between observed and permuted statistics. x must be measured entirely before launch, which consumption_pre is. Expected variance reduction is about 1 - corr(y, x)^2; measure the achieved reduction from the two null distributions rather than asserting it.
Worked solution 45 min
- cap_y = np.quantile(df.consumption_28d, 0.99); y = np.minimum(df.consumption_28d.to_numpy(float), cap_y); cap_x = np.quantile(df.consumption_pre, 0.99); x = np.minimum(df.consumption_pre.to_numpy(float), cap_x)
- t = (df.arm == 'treatment').to_numpy(); n1 = int(t.sum()); n0 = len(t) - n1; obs = y[t].mean() - y[~t].mean()
- rng = np.random.default_rng(11); L = np.tile(t.astype(np.int8), (20_000, 1)); rng.permuted(L, axis=1, out=L); s1 = L @ y; stats = s1/n1 - (y.sum() - s1)/n0
- p = (1 + int(np.sum(np.abs(stats) >= abs(obs)))) / (20_000 + 1)
- theta = np.cov(y, x, ddof=1)[0,1] / np.var(x, ddof=1); y_adj = y - theta*(x - x.mean()); repeat steps 2 to 4 on y_adj and compare stats.std(ddof=1) between the two runs.
Follow-up
- The p-value is 0.04 with the cap and 0.31 without it. What do you report, and what did you pre-register?
- Colleagues in a shared workspace can see the treated behaviour. How does that change the design and the estimate?
- How many accounts would you need to detect a 5% lift given this outcome's distribution?
Find paid seats that never called the API
dim_user_membership carries user_id, account_id, seat_type, is_service_account, deactivated_at and last_seen_at. fct_api_request carries user_id, which is null whenever the caller is a service account or an unattended key, plus account_id, request_at and http_status. For a single account, list every licensed_paid seat with is_service_account = false and deactivated_at null that issued no request at all in the trailing 90 days, returning user_id and last_seen_at. A first draft writes NOT IN against a subquery over fct_api_request.user_id. State exactly what that draft returns and why, then write the correct query.
Approach
- Read the column comment first: user_id is nullable on the fact, so the subquery almost certainly contains at least one NULL for any account that runs a service account or an unattended key.
- Work the three-valued logic out loud. x NOT IN (a, NULL) expands to NOT (x = a OR x = NULL); the second disjunct is UNKNOWN, so for any x not equal to a the whole predicate is UNKNOWN and the row is filtered out. The draft returns zero rows, which reads as full seat utilisation.
- Rewrite as NOT EXISTS with the 90-day and status predicates inside the correlated subquery. Putting them in the outer WHERE instead turns the anti-join into a different question and silently changes the answer.
- Correlate on both user_id and account_id. A membership is (user_id, account_id) and one person can hold memberships in several accounts, so correlating on user_id alone marks a seat as active because that human was busy somewhere else.
- Filter the seat side to seat_type = 'licensed_paid', is_service_account = false and deactivated_at IS NULL, and say what the resulting count means next to contracted_seats on the current subscription row.
Worked solution 15 min
- Confirm the hazard with one query: SELECT count(*) FROM fct_api_request WHERE user_id IS NULL AND account_id = :account_id. Any non-zero result proves the draft returns nothing.
- Write the seat side: SELECT user_id, last_seen_at FROM dim_user_membership WHERE account_id = :account_id AND seat_type = 'licensed_paid' AND NOT is_service_account AND deactivated_at IS NULL.
- Attach AND NOT EXISTS (SELECT 1 FROM fct_api_request r WHERE r.user_id = m.user_id AND r.account_id = m.account_id AND r.request_at >= now() - interval '90 days').
- Compare the row count against the same query written as a LEFT JOIN with a WHERE r.user_id IS NULL; the two must agree exactly.
- Divide the active seat count by contracted_seats from the account's current fct_subscription_period row and state the utilisation figure.
Follow-up
- Adding AND user_id IS NOT NULL to the subquery also fixes NOT IN. Why is NOT EXISTS still the form you would leave in the repository?
- last_seen_at looks like a shortcut for the whole query. What does it actually record, and where does it disagree with API activity?
- This is a seat-reduction risk list. What threshold would you attach before handing it to an account team, and what happens to seats below it?
Net revenue retention on a cohort frozen twelve months back
fct_subscription_period carries subscription_period_id, account_id, arr_cents, plan_code, term_start_date, term_end_date, booked_at, amendment_type, superseded_by_id (the subscription_period_id of the version that replaced this one, null on the live version of a lineage) and is_current. Compute net revenue retention for month M: the summed arr_cents at M for the set of accounts holding arr_cents > 0 at M-12, divided by that same set's arr_cents at M-12. An account can hold more than one live subscription, a churned account contributes zero rather than dropping out, and nothing signed after M-12 may enter either side. Return the ratio plus the expansion, contraction and churn components in cents.
Approach
- Write one reusable as-of ARR snapshot parameterised by a date: rows whose term brackets the date AND whose booked_at is at or before the date, then only the version of each lineage that is still live at that date, then sum arr_cents per account. The booked_at guard matters because an amendment signed in advance otherwise co-exists with the term it replaces and double counts the account.
- Collapse the lineage on superseded_by_id, not on any attribute of the contract. Keep a row when superseded_by_id IS NULL, or when the successor it points at was booked after the date. Ranking with ROW_NUMBER() OVER (PARTITION BY account_id, plan_code ...) instead is wrong in both directions: an amendment that moves the account from one plan_code to another puts the old and new versions in different partitions, so both are rank 1, both bracket the date, and the account's arr_cents is counted twice; and two genuinely concurrent subscriptions that happen to share a plan_code land in one partition, so one of them is deleted.
- Resolve the successor with a LEFT JOIN back to fct_subscription_period on subscription_period_id, and treat a missing successor as not superseded. An inner join would silently delete an account's ARR on a dangling pointer, which is a data-quality bug in the source, not a retention movement.
- Never use is_current for the M-12 side. is_current describes today; using it at the historical snapshot backdates the present contract onto last year's cohort and makes retention look like 100 percent by construction.
- Freeze the cohort from the M-12 snapshot where arr_cents > 0, then LEFT JOIN the M snapshot onto it and COALESCE the missing side to zero. An inner join deletes exactly the churned accounts, which is the single largest way this number gets overstated.
- Return a ratio of sums, not a mean of per-account ratios. The two are different estimands: contraction is floored at zero while expansion is unbounded, so the mean of ratios is both biased relative to the aggregate and far noisier on a skewed revenue base.
- Decompose per account on the delta: positive delta is expansion, negative delta with a non-zero M value is contraction, a zero M value is churn. The three components must reconcile to numerator minus denominator.
- Prove no leakage: any account whose first contract began after M-12 must be absent from both sides, and the cohort row count must be identical in the numerator and denominator.
Follow-up
- Net revenue retention can rise while the business shrinks. Show one mechanism and name the guardrail that catches it.
- How do you handle an account that co-terms two subscriptions into one mid-window, so the subscription count changes but the money does not?
- Finance computes this from invoiced amounts and gets a different number. Which is right for which question?
Given a dataset, how would you approach identifying trends and insight…
Given a dataset, how would you approach identifying trends and insights?
Approach
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How do you prioritize tasks when faced with multiple deadlines?
How do you prioritize tasks when faced with multiple deadlines?
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- State what result would change your recommendation, so the answer is falsifiable.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
How would you approach building a recommendation system for a new prod…
How would you approach building a recommendation system for a new product?
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- How would you detect that the metric is being gamed rather than genuinely improving?
Present a case where you had to analyze data without clear guidance.
Present a case where you had to analyze data without clear guidance.
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
Walk me through your thought process for designing an A/B test.
Walk me through your thought process for designing an A/B test.
Approach
- Decide the analysis before seeing data, including how long it runs and when you look.
- Name the randomisation unit first; it decides the variance and what the test can detect.
- State the primary metric and the minimum effect worth shipping, then size the test.
Follow-up
- What would you do if you could not randomise at all?
- How would you handle interference between treated and control units?
Explain the difference between supervised and unsupervised learning.
Explain the difference between supervised and unsupervised learning.
Approach
- Work from the decision backwards to the evidence you would need.
- State your assumptions explicitly before working the problem.
- Clarify what is being asked and what a complete answer would contain.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Build an activation metric tree for a self-serve onboarding rebuild
The self-serve signup flow is being rebuilt. You have dim_account (account_id, created_at, is_internal, first_workspace_at, is_current) and fct_api_request (account_id, workspace_id, environment, api_key_id, http_status, traffic_class, request_at). Build a three-layer metric tree from signup through to production dependency, name the primary metric for this specific change, and state the fixed measurement window and the denominator for each layer. Two of the three layers share a denominator and the third does not, so say which invariant still holds across the tree once the denominators differ. Say which layer the onboarding team can actually move and which it cannot, and why raw signup counts are not on the tree.
Approach
- Fix the cohort once: accounts created in cohort period P with is_internal = false, collapsed to one row per account_id since dim_account is type 2. Say out loud that employee, demo and load-test accounts are removed, otherwise the tree partly measures internal testing.
- Layer one, qualified signup, on the cohort denominator: accounts recording at least one successful request (http_status < 400) with traffic_class in ('interactive','batch') within 14 days of created_at. This replaces raw signup counts, which measure registration-form friction and duplicate sign-ups from one organisation rather than demand.
- Layer two, activation, also on the cohort denominator: the share of the cohort whose first successful request with a non-null api_key_id arrives within a fixed 168 hours of created_at. Fixed window, not trailing, so that cohort weeks are directly comparable and no cohort is censored.
- Layer three, production depth, on a conditional denominator: the share of activated accounts holding a workspace with environment = 'production' whose trailing 7-day count of successful non-synthetic requests exceeds a floor calibrated from the data, for example the volume above which 90-day retention stops rising. A floor picked by intuition makes this a metric about the floor. Conditioning on activation is the right operational choice, because the question it answers is whether accounts that got started go on to depend on the product, but it means this rate is not on the same scale as the two above it: a conditional rate of 0.50 sits perfectly happily underneath a 0.30 activation rate. To place it back on the funnel, multiply it by the activation rate and report that product as the cohort share.
- Align the traffic_class filter across all three layers, otherwise the underlying account sets do not nest: an activation definition that admits traffic_class = 'ci' will count accounts that layer one excluded, and the tree stops being a funnel. With mixed denominators the invariant that survives is on account sets and therefore on counts, not on rates. Then name the primary metric for this change, the seven-day activation rate, because the onboarding flow controls the path to first successful call and nothing below it. Production depth is the lagging check that the activation gain was real rather than one sample request.
Worked solution 25 min
- Build the cohort table: one row per account_id, bucketed by the ISO week of created_at, is_internal = false, dim_account collapsed to is_current = true.
- For each layer, find the first qualifying request per account from fct_api_request under that layer's filter and keep min(request_at), using EXISTS or a lateral so the cohort stays at one row per account.
- Compute layers one and two as shares of the cohort denominator and layer three as a share of the activated accounts, carrying the raw account count beside every rate, and mark any cohort week not yet clear of the longest window (14 days) as not reportable.
- Plot the three rates by cohort week, and separately confirm nesting on the counts within each cohort.
Follow-up
- Activation rises 6 points and the production-depth rate among activated accounts is flat 60 days later. Say what happened to the cohort share of production-depth accounts, and why a flat conditional rate is the better of the two readings available here.
- Median time-to-first-successful-call answers the same question with more information. Why is it a worse weekly dashboard metric, and what estimator would make it honest?
Monthly logo churn triples with no change in satisfaction
Monthly logo churn, computed as churned accounts divided by all paying accounts, tripled last month. Contracts are annual. From fct_subscription_period (account_id, term_start_date, term_end_date, amendment_type, auto_renew, booked_at, superseded_by_id, is_current) and dim_account (account_id, churned_at, account_status, employee_band, acquisition_channel), rebuild churn on the renewal-eligible base, separate calendar effects from customer behaviour, and state whether retention actually changed. Note that churned_at is sometimes set when the record was updated rather than at term end.
Approach
- Rebuild the denominator as accounts whose term_end_date falls in the month. An account eleven months from renewal sits in the current denominator while being structurally incapable of entering the numerator, so on annual contracts the published rate understates the truth by roughly the reciprocal of the annual renewal fraction and moves with the signing calendar.
- Plot the renewal-eligible base by month across two years. A signing surge twelve months earlier reproduces itself as an eligibility surge now, and a rate whose denominator ignores that tracks the sales calendar rather than customer sentiment.
- Date each churn by term_end_date, never by churned_at or an update timestamp. Inspect the distribution of churned_at minus term_end_date: a backlog cleared in one batch appears as a mass at a single date and shifts losses into whichever month the operations team did its paperwork.
- Apply the 45-day grace for late renewal paperwork so the most recent month and a half is marked not reportable, rather than printing a number that will rise once the paperwork lands.
- Compare corrected gross logo retention against its own trailing distribution, and if a real change survives, cut it by employee_band, acquisition_channel and plan_tier before proposing any cause.
Follow-up
- Some contracts in the window have not reached their renewal date. When does this require a survival estimator rather than a simple rate, and which one would you use?
- How do you report churn to an audience that wants a monthly number when the underlying event is annual and lumpy?
For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Design one test end to end on paper
- Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
- Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
- State in advance what you will do if the primary metric is flat while a secondary metric is significant.
Deliverable: A one-page test design with a decision rule written before launch.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02Power arithmetic until it is automatic
- Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
- Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
- Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.
Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.
Practice prompt ↗Practice prompt ↗Practice prompt ↗03Variance and the unit-of-analysis problem
- Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
- Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
- Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.
Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.
Practice prompt ↗Practice prompt ↗04Validity threats you can actually test for
- Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
- Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
- Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.
Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.
Practice prompt ↗Practice prompt ↗Worked solution ↗05When randomization is not available
- Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
- Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
- List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.
Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.
Practice prompt ↗Practice prompt ↗06The readout query
- Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
- Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
- Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.
Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.
Practice prompt ↗Practice prompt ↗07Present it to someone who will not read the appendix
- Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
- Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
- Rewrite your opening line so the recommendation lands before any methodology.
Deliverable: A one-page readout whose first line is the recommendation.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Saying no well is a senior skill and it is rarely rehearsed. Think of a time you told someone their analysis was not worth doing, or that the experiment could not answer their question at the sample size available. Explain what you offered instead. Refusal without an alternative reads as obstruction rather than judgement.
Tell me about a challenging project you worked on. What was your role,…
Tell me about a challenging project you worked on. What was your role, and what did you learn from the experience?
Approach
- Quantify the outcome, including what you would not claim credit for.
- Name the disagreement or constraint, and how you resolved it with evidence.
- Close with what you would do differently, concretely.
Follow-up
- What did you decide not to do, and why?
- How did you know the outcome was caused by your change?
Scope an open-ended request to predict account churn
A customer success director asks for a list of accounts about to churn. You know only that the team has six people and that contracts are annual. Available data is fct_subscription_period, fct_usage_daily, fct_api_request, fct_support_ticket and dim_account. Before writing any code, produce the questions you need answered, a proposed definition of about to churn, and the shape of the artefact you would hand back, including the operating point that turns a score into a decision.
Approach
- The interviewer is probing whether you convert a vague request into a decision with a capacity constraint attached. A candidate who starts talking about model families has already failed the exercise.
- Pin the event and the horizon first. Churn is only possible at term_end_date, so the population is accounts renewing in the next 60 to 90 days, not the whole base. Ask explicitly whether contraction and downgrade count as churn or only full non-renewal, because the three have different base rates and different interventions.
- Pin the action and the capacity. Six people times a realistic number of meaningful interventions per week gives k, and k is what the list is ranked to. Evaluate on precision at k rather than a global AUC over accounts that will never be contacted.
- Audit leakage before choosing features. Every feature needs a timestamp proving it existed before the prediction date. A downgrade amendment, a churn reason code, and a ticket opened after the renewal conversation started are all leaks that will make the offline number look excellent and the live list useless.
- Ask for the counterfactual now rather than later. Coverage is assigned deliberately, so without a held-out slice agreed at the start the intervention can never be evaluated, and you will be asked for its impact in nine months regardless.
- Propose the smallest artefact that closes the loop: a weekly ranked list sized to capacity with two or three inspectable reasons per row, plus a stated policy for accounts below the line.
Follow-up
- The director insists all accounts are in scope, not only those renewing soon. How do you answer without simply refusing?
- Historical non-renewals number about 30 a year. At what point do you tell them a model is the wrong tool and a rules list is better?
- Which candidate features would you drop purely because you cannot date them?
Allocate one analyst week across three competing requests
Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.
Approach
- The interviewer is probing whether you prioritise on decision timing and reversibility or on who asked most forcefully. Sort by the date each decision is actually taken and by what the default outcome is if nothing arrives.
- Apply that sort concretely. The Thursday readout has a hard irreversible deadline and no value afterwards. The pricing review has three weeks of slack. The renewal list has a rolling deadline set by term_end_date, so part of it is urgent this week and the rest is not, which means it can be split rather than deferred whole.
- Find the cheapest sufficient version of each request rather than the full version. The readout goes in full. The renewal list ships as a filtered query over renewal-eligible accounts ranked by two inspectable signals rather than as a model. The margin work is scoped to the accounts that dominate the pricing decision, since revenue is heavily skewed and the tail will not change the conclusion.
- Make the trade visible in one written note to all three at once, with dates. Telling each person separately that they are the priority is how an allocation becomes a credibility problem.
- Refuse something explicitly and say why. The model version of the renewal list is the usual candidate, because it cannot be evaluated without a holdout nobody has agreed to yet, and building it this week forecloses that.
- Leave slack. A plan with none is a plan to miss the one deadline that cannot move.
Follow-up
- The sales leader escalates to your manager. What did you already do that makes that a short conversation?
- Which of the three deadlines would you push back on, and what exactly would you ask for?
- What would you change about how these requests reach you so next week is not the same?
- 01
Tell me about a challenging project you worked on. What was your role, and what did you learn from the experience?
- 02
A customer success director asks for a list of accounts about to churn. You know only that the team has six people and that contracts are annual. Available data is fct_subscription_period, fct_usage_daily, fct_api_request, fct_support_ticket and dim_account. Before writing any code, produce the questions you need answered, a proposed definition of about to churn, and the shape of the artefact you would hand back, including the operating point that turns a score into a decision.
- 03
Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.
Is this an official Protagonist interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Protagonist. Rounds and questions reflect what candidates have reported, not a process Protagonist has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult is the interview process for the Data Scientist position?
The interview process can be challenging, requiring thorough preparation in technical areas as well as soft skills. Candidates typically find that a focused approach to understanding both the role and the company culture can significantly enhance their performance.
PracHub interview research ↗What differentiates successful candidates from others?
Successful candidates often exhibit strong technical proficiency, a structured problem-solving approach, and excellent communication skills. They are also able to align their experiences with Protagonist's values and demonstrate a collaborative mindset.
PracHub interview research ↗What is the typical timeline from the initial screen to offer?
The timeline can vary but generally spans several weeks. Expect to engage in multiple rounds of interviews, including technical screenings and behavioral assessments, before receiving an offer.
PracHub interview research ↗How does the company approach remote work?
Protagonist embraces a flexible work environment, with options for remote and hybrid work arrangements. Candidates should be prepared to discuss their preferences during the interview.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
[24]7.ai Data Scientist Interview Questions & Guide 2026
Modeling drills for [24]7.ai data scientist prep: choosing a baseline, reading regression coefficients honestly, handling class imbalance, and defending a model you would actually ship.
Xcellent Technology Solutions Data Scientist Interview Questions & Guide 2026
Modeling drills for Xcellent Technology Solutions data scientist prep: choosing a baseline, reading regression coefficients honestly, handling class imbalance, and defending a model you would actually ship.
Wolters Kluwer Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for Wolters Kluwer data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.
Zscaler Data Scientist Interview Questions & Guide 2026
Data-quality drills for Zscaler data scientist prep: survivorship and selection bias, leakage through time-ordered features, and extreme values where provenance decides the call. A logging fault or unit slip gets corrected or dropped; a gen