The work in general product and business analytics is mostly deciding what a number should mean and then defending that decision. A typical week contains a metric that moved and nobody can explain, an experiment readout where the headline is flat but three segments look interesting, a request to define success for a feature that ships next month, and a forecast or sizing input for a planning cycle. Very little of it arrives as a well-posed question; the first deliverable is usually a restatement of the question in terms that have a numerator, a denominator and a window. The output is a decision about shipping, pricing, spending or stopping, and it has to survive somebody else recomputing it from the same tables.
The skills that carry the most weight are SQL across mismatched grains, experiment design, and the judgement to say what a result does not support. Event, session, subscription-period and exposure data all live at different grains, and the standard failure is a join that fans revenue across rows or an inner join that deletes exactly the population the question was about. Experiment work is judged less on whether a test statistic was computed correctly than on whether the randomisation unit matched the interference structure, whether the horizon was fixed in advance, and whether the analysis population was the exposed set rather than the assigned set. Above all of this sits communication: a correct analysis delivered as a table of numbers with no stated decision is treated as unfinished.
What makes it hard is the absence of a physical constraint to anchor definitions. A supply chain has units that either arrived or did not; a product funnel has whatever the tracking plan happened to record, sessionised by a rule somebody chose, joined on an identity that resets when a browser clears its cookies. Two competent analysts can produce different conversion rates from the same warehouse and both be right, which means the defensible answer is the one whose assumptions are written down. On top of that, the effects being measured are small: a genuine improvement to a mature funnel is often a fraction of a percentage point in absolute terms, comfortably inside the noise of a week, so most of the skill is in variance control and sizing rather than in modelling.
Metric definition and metric-tree construction
editorialPreparation area covering the translation of a loosely worded goal into a metric with a stated numerator, denominator, window and exclusion rule, and the assembly of those metrics into a tree where each layer explains the one above it.
What to demonstrate
- Whether a proposed metric is actually computable from the available tables, and whether the candidate says so when it is not
- Whether the candidate names the window, the lag before publication and the excluded populations rather than leaving them implied
- Ability to explain how a metric would be gamed and to name the specific guardrail that catches that behaviour
- Judgement about the grain a metric belongs at, particularly the distinction between a per-user and a per-account question
How to prepare
- Write out each metric in the tree above from memory, including its lag rule, until the definitions are recall rather than reconstruction
- Take three vague goals such as 'improve engagement' and produce two defensible metrics for each, with the argument for choosing between them
- Practise stating the failure mode of a ratio metric whose denominator is the thing that moved, and how you would detect that in the data
- Rehearse the difference between a count, a rate and a rate-of-rates, and which of the three survives being rolled up across periods
Warehouse SQL across event, session, subscription and exposure grains
editorialPreparation area for queries that span an append-only event log, derived sessions, billing periods and experiment exposures, where a naive join fans out revenue and an inner join removes exactly the rows that carry the answer.
What to demonstrate
- Correct handling of fan-out: pre-aggregating one side in a CTE before joining, and being able to demonstrate what the unaggregated join would have done to the total
- Deliberate treatment of NULL foreign keys, including signed-out events with no user_id, sessions with consent_state = 'denied' and accounts with no subscription row
- Fluency with window functions for first-event identification, cohort indexing, running counts and deduplicating overlapping billing periods
- Whether rates are rolled up by re-summing numerator and denominator rather than by averaging daily or weekly rates
How to prepare
- Build the full visitor-to-signup-to-activation funnel as one query using left joins forward, so that lost stages stay visible as NULLs instead of disappearing
- Write MRR at the start and end of a month from fct_subscription_period, handling mid-period plan changes without double-counting the account
- Practise a retention triangle from dim_user and fct_event with both window bounds enforced, and verify the row count at each grain before aggregating
- Reconstruct a per-attempt funnel from flow_instance_id, and be able to say what breaks when that id is missing
Experiment design, sizing and readout discipline
editorialPreparation area for choosing a randomisation unit, powering a test before it launches, and reading it out in a way that survives scrutiny, including the cases where the honest answer is that the test could not have detected the effect in question.
What to demonstrate
- Whether the randomisation unit matches the interference structure, and whether the standard error is computed on that unit rather than on the row
- Ability to size a test from first principles: for a proportion, roughly 16 p(1-p) divided by the squared absolute lift per arm, at 5% two-sided alpha and 80% power with equal arms
- Recognition that the analysis population is the exposed set, that assignment without exposure dilutes the estimate toward zero, and that a sample-ratio mismatch invalidates the readout rather than being a footnote
- Whether novelty and primacy effects are addressed, and whether the candidate distinguishes a metric that recovered from one that was never affected
How to prepare
- Memorise the proportion sizing shortcut and practise computing a minimum detectable effect from a realistic weekly traffic figure in under a minute
- Be able to state the design effect for clustered randomisation, variance inflated by 1 + (m-1) rho with m the mean cluster size and rho the intraclass correlation, and what it does to required sample size
- Practise explaining why a per-session metric randomised on users needs a delta-method or bootstrap standard error rather than a naive one
- Prepare a short account of variance reduction with a pre-period covariate, including the fact that it removes a share of variance equal to the squared correlation between covariate and outcome
Causal inference when randomisation is unavailable
editorialPreparation area for questions where an experiment is impossible, illegal or already too late, covering the quasi-experimental designs, the assumption each one rests on, and how to argue that assumption is plausible in the case at hand.
What to demonstrate
- Whether the candidate names the identifying assumption rather than the method, for instance parallel counterfactual trends rather than 'difference-in-differences'
- Ability to choose between difference-in-differences, a geo or time-based holdout, regression discontinuity and matching on the basis of what the data actually supports
- Awareness of what matching cannot fix, namely selection on anything unobserved, and willingness to say so instead of presenting a balanced covariate table as proof
- Whether a placebo or pre-trend check is proposed before the estimate is believed
How to prepare
- Practise writing the parallel-trends check you would run before a difference-in-differences estimate, and what you would conclude if the pre-period trends diverge
- Prepare one worked case where a launch was staggered across regions and explain how the staggered timing itself provides the comparison
- Rehearse the argument for a geo holdout as the cheapest credible read on channel incrementality, including how you would size it on total conversions
- Be able to state in two sentences why observational channel comparisons overstate the channels closest to conversion
Diagnosing a metric movement and landing the decision
editorialPreparation area for the open-ended case where a number moved and the task is to find out why and say what to do, working from a decomposition rather than from a list of guesses.
What to demonstrate
- Whether the first move is to verify the metric itself, covering instrumentation changes, a late-arriving data partition, a client release and a definition edit, before any product hypothesis
- Whether the decomposition is systematic, splitting the metric into its factors and then across segment, platform, geography, tenure and channel rather than sampling hypotheses at random
- Ability to size the contribution of each candidate cause, so that a segment moving sharply but carrying two percent of volume is correctly set aside
- Whether the answer ends in a recommendation with its uncertainty stated, and whether the candidate says plainly when the data cannot yet support one
How to prepare
- Rehearse a fixed opening sequence: confirm the movement is real and outside normal weekly variation, confirm the pipeline and tracking plan did not change, then decompose
- Practise the multiplicative decomposition of a funnel metric so that a drop can be attributed to a specific stage rather than to the funnel in general
- Build the habit of quantifying every candidate cause in the units of the headline metric, so contributions can be ranked and the remainder made explicit
- Practise delivering the finding in three lines, the movement, the cause with its share, and the decision, before any supporting detail
2 candidate reports. Individual accounts describe a particular role and hiring cycle.
Zomato Software Engineer interview with paper coding
The Zomato process began with resume shortlisting and moved straight to technical interviews. The first technical round mixed DSA, CS fundamentals, puzzle-style questions, resume details, and basic web development. That combination initially threw me because it was not a pure coding interview. The next technical round made the DSA work more intense, and I had to write code on paper rather than on…
Read full experienceZomato Software Engineer interview: a second recruiting chance
My Zomato experience was a rollercoaster. I was initially rejected, apparently because I did not have the years of experience they wanted, but a recruiter contacted me again a few weeks later with another opportunity from a different team. A team lead later became involved. The second path felt like a fresh chance that followed a similar recruiting rhythm, rather than a completely unrelated proce…
Read full experiencePracHub editorial advice for the preparation topics above.
Counting on an identity key that changes underneath the metric
visitor_id is per browser and per device, and it resets on cookie clearance, private browsing and platform privacy changes, so the distinct-visitor count drifts upward for reasons unrelated to reach. Any rate with visitors in the denominator therefore decays over time even when behaviour is constant, and any rate with visitors in the numerator inflates. The stitching at signup makes it worse in both directions: a user who signed up on mobile and returns on desktop is two visitors and one user, while a shared device is one visitor and several users. Decide which key each metric is counted on, write it into the definition, and when comparing a period before and after a platform privacy change, expect a level shift in every visitor-keyed metric and do not attribute it to the product.
Watching an experiment daily and stopping when it crosses significance
A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.
Interpreting a change before checking data quality and logging
Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.
Accepting a metric definition without asking about the denominator
Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Audit a one-day event extract for structural defects
You receive a one-day extract of fct_event as a DataFrame with event_id, occurred_at_utc, received_at_utc, visitor_id, user_id, account_id, event_name, is_bot_flagged and surface. Write a function returning one row per data-quality rule with the rule name, the failing row count and the failing share of the extract. Cover at minimum: duplicate event_id, received_at_utc earlier than occurred_at_utc, occurred_at_utc later than the extract's maximum received_at_utc, account_id present while user_id is NULL, and rows whose occurred_at date differs from their received_at date. Do not drop rows; report only.
Approach
- Compute the extract's own reference clock first: max(received_at_utc). Wall-clock now() is wrong here because the extract may be replayed days later, which would turn every row into a future-dated failure.
- Express each rule as a boolean Series over the same index so the checks compose, then aggregate with .sum() and divide by len(df). Building a list of (name, mask) pairs keeps the rule set extensible and keeps one code path for counting.
- For the duplicate rule, decide and state the convention: df.duplicated('event_id', keep=False).sum() counts every member of a duplicated group, df.duplicated('event_id').sum() counts only the surplus copies. Either is defensible; an unstated choice is not. The rest of this item assumes keep=False.
- Treat received_at < occurred_at as clock skew, not corruption: occurred_at is client-supplied. Separate it from the date-mismatch rule, which is the one that actually breaks a daily metric keyed on occurred_at.
- Know which rules imply which before you read the counts. A row whose occurred_at exceeds max(received_at_utc) has its own received_at no later than that maximum, so it is necessarily a clock-skew row as well: the future-dated mask is a subset of the skew mask, always. Neither is a subset of the date-mismatch mask, because skew of a few minutes inside one UTC date mismatches nothing.
- Return a tidy DataFrame sorted by failing_share descending, and add a boolean column saying whether the rule should block publication, so the output is a decision rather than a list of numbers.
Follow-up
- The date-mismatch count is 2.1 percent on this extract. What late-arrival rule would you write for a daily metric, and how many days would you hold the number open?
- Duplicate event_id values appear only on the 'core_action_completed' event. What upstream cause would you check before deduplicating?
- Which of these rules should fire an alert at the pipeline, and which should only appear in a weekly review?
Sessionise an event stream with gap and midnight rules
Sessionise a raw event stream. Input: a DataFrame with visitor_id, user_id (often NULL), occurred_at_utc and event_name, unsorted, up to 5 million rows. A session breaks when the gap from that visitor's previous event exceeds 30 minutes, and is force-closed at UTC midnight so no session spans two calendar dates. A gap of exactly 30 minutes does not break. Emit one row per session with session_id, visitor_id, the user_id as of the last event in the session, started_at_utc, ended_at_utc, session_date, duration_seconds and event_count. Vectorise; do not loop per visitor.
Approach
- Sort by ['visitor_id', 'occurred_at_utc', 'event_id'] once, then express the whole problem as one boolean vector: a row starts a new session when the visitor changed, or the gap exceeds 30 minutes, or the UTC date differs from the previous row's UTC date. Cumsum that vector and you have the session key.
- Get the comparison direction right on the gap: the rule is strictly greater than 1800 seconds, so an event at exactly 1800 seconds continues the session. Write it as gap > pd.Timedelta(minutes=30), and make the tie a test case rather than an assumption.
- Derive the midnight break from the date change, not from inserting synthetic boundary rows. A date change implies a break even when the gap is two seconds, which is precisely the force-close rule and is why the two conditions are ORed rather than one subsuming the other.
- Aggregate with a single groupby on the session key: min and max of occurred_at_utc, size for event_count, and last for user_id, which is correct because the frame is already sorted so 'last' is the final event in the session. That is the identity-as-of-session-end rule.
- Compute duration_seconds as (max - min).dt.total_seconds(), which makes a single-event session 0 seconds. Say so explicitly, because a downstream mean session duration is sensitive to whether single-event sessions are 0 or excluded.
Follow-up
- Sessions are used as the denominator of a conversion rate. How does moving the inactivity gap from 30 to 45 minutes move that rate, and in which direction?
- A visitor's clock is 40 minutes ahead, so their events arrive with future occurred_at values. What does your sessioniser do, and what would you rather it did?
- The same person signs up mid-session on mobile and continues on desktop. How many sessions and how many users does your output show, and is that the right answer?
Rebuild per-visitor ordering without groupby convenience methods
You have a DataFrame of 2 million fct_event rows with visitor_id, occurred_at_utc and event_id, unsorted and containing duplicate timestamps within a visitor. Produce three new columns: event_rank, the 1-based position of the event within its visitor ordered by occurred_at_utc; seconds_since_prev, the gap to that visitor's previous event, NULL for the first; and is_first_for_visitor. You may use sort_values, shift, cumsum, numpy and boolean masking. You may not use groupby.transform, groupby.apply, groupby.cumcount, groupby.rank or merge_asof. Break timestamp ties on event_id.
Approach
- Sort once by ['visitor_id', 'occurred_at_utc', 'event_id'] and reset the index. The whole exercise reduces to row arithmetic on a sorted frame, and the tiebreak on event_id is what makes the result reproducible across runs.
- Mark visitor boundaries with is_first = df['visitor_id'].ne(df['visitor_id'].shift()). This is the single fact every other column derives from.
- Compute seconds_since_prev as the diff of the timestamp column, then overwrite it with NaT/NaN wherever is_first is True. The shift crosses the boundary between visitors and will otherwise hand the first row of each visitor the last event of the previous one.
- Build event_rank from a running counter that resets at boundaries: take a global cumulative position (np.arange(len(df))) and subtract, per row, the global position at which that visitor started. Get the start position by forward-filling the positions where is_first is True, which is a cumsum-free reset and is O(n).
- Verify against the forbidden method once, as a test rather than as the implementation, and confirm the two agree on every row.
Worked solution 20 min
- Sort on the three-key tuple and reset_index(drop=True).
- Compute is_first via .ne(.shift()), which is True for row 0 because the shifted value is NaN.
- pos = np.arange(len(df)); start = pd.Series(np.where(is_first, pos, np.nan)).ffill(); event_rank = (pos - start + 1).astype(int).
- gap = df['occurred_at_utc'].diff().dt.total_seconds(); gap[is_first] = np.nan.
- Assert event_rank equals df.groupby('visitor_id').cumcount() + 1 on the sorted frame.
Follow-up
- The frame does not fit in memory. How does your approach change if you can only process one visitor-partitioned chunk at a time?
- occurred_at_utc is client-supplied and sometimes runs backwards within a visitor. Does your seconds_since_prev go negative, and should it?
- How would you extend this to reset the counter at every change of surface as well as visitor?
Pair flow starts to completions within a thirty-minute bound
fct_event holds event_id, flow_id, flow_instance_id, event_name, occurred_at_utc, surface, app_version. Using only rows where event_name is 'flow_started', 'flow_completed' or 'error_shown', compute the daily core-flow completion rate by surface and app_version. Numerator: distinct flow_instance_id whose flow_completed occurs within 30 minutes of its flow_started with no error_shown for the same instance in between. Denominator: distinct flow_instance_id with a flow_started that day. Report rows with a NULL flow_instance_id as a separate coverage count, not inside either side.
Approach
- Collapse the stream to one row per flow_instance_id using conditional aggregation: MIN(occurred_at_utc) FILTER (WHERE event_name = 'flow_started') AS started, MIN(...) FILTER (WHERE event_name = 'flow_completed') AS completed, MIN(...) FILTER (WHERE event_name = 'error_shown') AS first_error. This makes the 'first' semantics explicit and avoids a self-join entirely.
- Apply the predicates on that single row: completed IS NOT NULL AND completed <= started + interval '30 minutes' AND (first_error IS NULL OR first_error > completed). Note the last clause encodes 'in between' literally, so an error surfaced after a successful completion does not disqualify the attempt.
- Bucket by started::date, surface and app_version. Splitting by app_version is not decoration: a completion-rate regression is nearly always confined to one client build, and a pooled daily rate hides it behind the installed base.
- Count NULL flow_instance_id rows as a separate coverage figure. They cannot be attributed to an attempt at all, so they belong in neither numerator nor denominator, and their share tells you how much of the rate is unmeasurable.
- For any roll-up to week or to all surfaces, re-sum the numerator and denominator rather than averaging the daily rates.
Worked solution 25 min
- Write the per-instance collapse CTE and confirm COUNT(*) equals COUNT(DISTINCT flow_instance_id).
- Add the three predicates one at a time, recording how many instances each removes.
- Aggregate by day, surface and app_version with FILTER on the clean-completion flag.
- Compute the NULL-instance coverage count as a separate select and report it alongside.
- Pick the worst day-surface-version cell and pull ten raw instances to confirm the disqualification reason.
Follow-up
- occurred_at_utc is client-supplied. What do you do with an instance whose completion timestamp precedes its start?
- flow_instance_id is missing on 4 percent of starts for one app_version. What can and cannot you conclude about that build?
- An instance starts at 23:55 and completes at 00:04. Which day owns it, and what does the alternative do to the daily series?
Paying accounts with no active seat in 28 days
dim_account holds account_id, account_type, lifecycle_status, seats_licensed. fct_event holds account_id, user_id, occurred_at_utc, is_core_action, and its account_id is NULL for every signed-out and pre-signup event. Find accounts with lifecycle_status = 'active' and account_type <> 'internal' that had no distinct user complete a core action in the trailing 28 days. Return account_id, seats_licensed and days since that account's most recent core action, with NULL where the account has never emitted one. Order by seats_licensed descending.
Approach
- Build the recent-activity set first: fct_event rows with is_core_action = TRUE, occurred_at_utc >= now() - interval '28 days', and an explicit account_id IS NOT NULL. Making the NULL exclusion explicit in the CTE is what lets you reason about the anti-join afterwards.
- Express the exclusion with NOT EXISTS (correlated on account_id) or a LEFT JOIN with an IS NULL guard. Do not use NOT IN against this column: it is nullable, and SQL's three-valued logic turns the whole predicate UNKNOWN, returning zero rows.
- Compute last-seen separately as MAX(occurred_at_utc) per account over all history, LEFT JOINed on, so an account that has never emitted a core action (NULL) is distinguishable from one that went quiet six weeks ago. Those two cases have different causes and different owners.
- Rank by seats_licensed, or better by the account's current mrr_cents_constant_fx if you are allowed the subscription table, because a silent fifty-seat account is a renewal conversation and a silent one-seat account is noise.
- Before shipping, check whether the never-seen group is a cluster by signup date or surface. A block of accounts with no events at all is usually an instrumentation gap, not a set of customers who stopped using the product.
Follow-up
- How would you distinguish a genuinely idle account from one whose events lost their account_id after an instrumentation change?
- Would you count on fct_event.account_id or resolve user_id through dim_user instead, and what does each choice miss?
- Licensed-seat utilisation is the continuous version of this. How would you turn this boolean into that ratio?
Choose one success number for a homepage redesign
A homepage redesign is ready to test. Marketing wants visit-to-signup conversion as the single success metric: distinct fct_session.visitor_id with a 'signup_completed' event in fct_event, over distinct visitor_id with a session started in the window where is_bot_flagged = FALSE and consent_state <> 'denied'. Sessions carry referrer_channel and device_type. Argue for or against that metric, name what you would decide on instead, and give the one guardrail you would refuse to ship without. Deliverable: the metric, the guardrail, and the failure mode in writing.
Approach
- List everything that moves this rate without the page changing: referrer_channel mix, device_type mix, cookie lifetime and bot-rule changes, then pre-register the segments you will decompose on so the decomposition is not chosen after seeing the result.
- Argue that signup is an intermediate outcome the homepage can inflate by over-promising, and propose activated signups per 1,000 eligible visitors — the same numerator further restricted to users clearing the seven-day activation bar — as the number the decision actually rests on.
- Explain the consent exclusion rather than copying it: a 'denied' session can never be joined forward to a user, so leaving it in the denominator puts visitors there who have no path into the numerator and depresses the level permanently.
- Separate what the metric can and cannot be used for: inside a randomised comparison over one window it is fine, but as a trend line across a platform privacy change it will step down because visitor_id resets more often, with no behaviour change behind it.
- State the trade you would accept in advance: a smaller conversion gain with flat activation beats a larger conversion gain with activation down, and write the threshold before the readout.
Follow-up
- The redesign wins overall but the entire gain sits in paid social. What do you do, and what would change your mind?
- How do you roll four weekly conversion rates up to a month, and why does the obvious way give a different answer?
- Where does holding the segment mix fixed stop working as a correction?
Define success for a rebuilt first-run onboarding checklist
A rebuilt first-run checklist ships to all new signups next month. You have dim_user (user_id, account_created_at_utc, signup_surface, is_internal) and fct_event (user_id, occurred_at_utc, is_core_action, event_name, surface). Propose a primary metric with an explicit numerator, denominator, window and publication lag, plus two guardrails and one diagnostic you would refuse to treat as success. The team wants a number it can read weekly, and the constraint is that everything must be computable from those two tables alone. Deliver the metric tree from the north star down to the metric you chose.
Approach
- Fix the grain and the cohort key first: the checklist is seen by users, so count on user_id with account_created_at_utc as the cohort key, and say out loud that the north star is account-grained so the tree crosses grains here deliberately rather than by accident.
- Take seven-day activation as the primary — core action on at least 2 distinct UTC dates inside [account_created_at_utc, +7 days) — because the two-distinct-days predicate cannot be satisfied by the single checklist-completion click the feature itself produces.
- Demote checklist completion rate to a diagnostic and give the reason: it is an output of the feature, so it is near-perfectly correlated with having shipped the feature and cannot fall when the feature is bad.
- Pick guardrails by the failure each one catches, not by what is easy to query: week-4 signup-cohort retention catches an activation gain that does not persist, and p50 minutes to first core action catches a checklist that adds steps to a path users already completed.
- State the operational rules explicitly: is_internal = FALSE, an 8-day publication lag, and signup_surface values 'invite' and 'sso_provisioned' split out because those users arrive through an administrator rather than a self-serve signup and may not be shown the checklist at all.
Follow-up
- Activation rises three points but week-4 retention is flat. What do you tell the team, and what would you need to distinguish a real gain from pulled-forward activity?
- The checklist ships on web only. What changes in the denominator, and what breaks if you leave every surface in?
- Signup mix shifted toward 'invite' the same week. How would you show whether the activation move was mix or behaviour?
A test that won on day four and lost by day twelve
A two-week test was checked every morning. On day four the primary metric crossed p = 0.03 and the team shipped. By day twelve, with logging still running, the estimate had fallen to a 0.2% lift at p = 0.61. The day-four deck also reported a significant 9% lift among paid_search visitors on mobile_web, one of twenty segments inspected. Explain both patterns quantitatively, say what the day-four evidence was actually worth, and specify the monitoring rule that prevents a repeat.
Approach
- Price the peeking. A fixed-sample test controls type I error at one pre-declared look. Ten daily looks at a nominal 5% raise the false positive rate to roughly 0.15 to 0.20, because the running test statistic takes a random walk that will eventually cross the boundary under the null.
- Price the winner's curse separately, because it is a different error. Conditioning an estimate on having crossed the threshold biases it away from zero, by a factor of roughly 3 when true power is 20% and only about 1.1 at 80% power. Day four sits at roughly 29% of planned information, which is exactly the low-power case.
- Read day twelve as the better estimate rather than as a contradiction. Nothing regressed; the day-four value was an extreme draw that was selected precisely for being extreme.
- Price the segment claim. Across twenty segments at a nominal 5%, at least one crosses about 64% of the time under a global null, since 1 - 0.95^20 = 0.64, and the slice that surfaces is by construction the one with the most favourable noise.
- Specify the rule instead of scolding. Either fix the horizon and look once, or use group-sequential boundaries such as O'Brien-Fleming that spend almost no alpha early, or use always-valid confidence sequences that stay correct under continuous monitoring. Pre-register the small set of segments, correct across the ones tested, and treat anything found afterwards as a hypothesis needing its own powered test.
Worked solution 25 min
- Count the looks actually taken, twelve daily, and state the inflated alpha band of 0.15 to 0.20 for ten looks, rising with more.
- Place day four on the information fraction, about 4/14 = 29% of planned sample, and apply the exaggeration factor for low power.
- Compute 1 - 0.95^k for the number of segments actually inspected; with k = 20 this is 0.64.
- Report the day-twelve interval as the estimate and say plainly that the experiment supports no effect at any size the team would ship for.
- Write the rule into the experiment template: horizon and segments declared at launch, sequential boundaries if interim looks are needed at all.
Follow-up
- Under O'Brien-Fleming with five planned looks, roughly what nominal p-value would the first look have needed to stop?
- The feature already shipped. What analysis can you still run, and what can it not tell you?
- If mobile_web paid_search were genuinely the only affected segment, how would you design the confirmatory test and how large would it have to be?
Separate a definition edit from a behaviour change
Seven-day activation rate is documented as core actions on at least 2 distinct UTC dates inside the first seven days after account_created_at_utc, and every cohort published through last week was computed on that definition. The dashboard now reads 38% for the signup cohort that started Monday, against 31% for the cohort before it. Two things landed in that week: an onboarding change, and a commit to the metric's SQL that you have not read yet and that came with no documentation update. You have dim_user, fct_event and the metric's version history. Deliver two numbers: how many of the 7 points are behaviour and how many are the definition. Then say which published cohorts are no longer comparable.
Approach
- Diff the metric SQL across the commit boundary and enumerate every changed predicate individually: the distinct-day threshold, the is_internal exclusion, the is_core_action source, the window bounds, and the identity key. A single commit routinely changes more than one, and each has its own sign.
- Dual-run both definitions over the same 26 cohort weeks. The definition effect is the level shift between the two series computed on identical data, which isolates it from anything that happened in the world.
- Read the behaviour effect only from the old-definition series, comparing the new cohort to its own trailing cohorts. The new-definition series cannot answer the behaviour question because it has no pre-period.
- Explain the mechanism of the changed predicate rather than only its size. Lowering a distinct-day threshold from 2 to 1 admits every user who acted once in a single session, which is a large and low-intent population, so a several-point jump is the expected magnitude and not a surprise.
- State the restatement plan: pick one definition, backfill the full history on it, and annotate the break date. A series carrying two definitions is not a time series.
Follow-up
- The onboarding team wants the 1-day threshold kept because it is easier to move. What is your argument, and what would change your mind?
- How would you make a definition change visible to every consumer of this metric without relying on people reading a changelog?
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Practice prompt ↗Practice prompt ↗Worked solution ↗02Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Practice prompt ↗Practice prompt ↗03Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Practice prompt ↗Practice prompt ↗Worked solution ↗04The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Practice prompt ↗Practice prompt ↗05Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Practice prompt ↗Practice prompt ↗Worked solution ↗06Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Practice prompt ↗07Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.
Quantify your own impact without claiming the topline you touched
You are writing the impact section of your own review. Over the year you ran four experiments, one of which shipped and three of which were flat; you corrected the definition of gross monthly revenue churn so that cancellation is recognised at period_end_utc; and you built a self-serve funnel dashboard. Weekly active accounts rose 14% over the same period. Your reviewer knows the data well. Write the three impact claims you would defend, stating for each what you contributed, what evidence supports it, and what portion of the outcome you are not claiming.
Approach
- Recognise what is being probed: whether you apply to your own work the causal standard you would apply to somebody else's roadmap claim. Nearly everyone who would reject 'accounts that do Y retain better' will write 'I drove a 14% increase' without noticing it is the same error with a friendlier subject.
- Sort the work by the kind of evidence it can carry. The shipped experiment is the only item with a randomised estimate, so it is the only one where an effect size is defensible, and you claim the interval rather than the point estimate.
- Claim the three flat experiments as decisions prevented and price them. Features not built, or built differently, on evidence, with the engineering weeks reallocated as the number somebody else can verify. A defensible null is a delivered decision and should be written as one.
- Claim the definition fix as correctness, not as improvement. The old figure was overstated by a specific percentage and appeared in a specific set of recurring documents; the impact is the change it produced in the forecast built on top of it, not a change in churn itself.
- Claim the dashboard on usage and displacement: distinct weekly users of it, and the ad-hoc request count for six months before against six months after. If the request log does not exist, record the claim as unverified rather than estimating it upward.
- Disclaim the 14% explicitly and once. State that it cannot be separated from seasonality, other teams' launches and a pricing change, and bound your own contribution from above using the shipped experiment's interval converted into headline units.
Follow-up
- Your shipped experiment's interval was +0.2pp to +1.4pp on activation. How much of the 14% can that account for, and how do you say so without undercutting yourself?
- A peer in the same cycle claims the full 14%. What, if anything, do you do about it?
- If you could only keep two of your three claims, which do you drop, and why that one?
Defend a flat experiment readout against a post-hoc segment
A feature you evaluated is flat on seven-day activation: +0.05pp with a 95% interval of [-0.47pp, +0.57pp], from 61,000 exposed users per arm in fct_experiment_exposure joined to dim_user and fct_event. Baseline activation is 32%. The launch team asks you to drop every surface except mobile_web, where the point estimate is +1.1pp, and re-run. You have ten minutes in their planning meeting. Deliver a spoken position: what you will and will not do, and the decision you recommend.
Approach
- Recognise what is being probed: whether you hold a statistical position under social pressure without becoming either rigid or apologetic. A generic answer says the segment is not significant; a strong one separates the request into a question that is answerable (is the mobile_web number real?) and one that is not (can we ship on it?), and answers both.
- Price the multiplicity out loud. The slice was chosen after seeing the results, so its estimate is selected on favourable noise and is biased away from zero. With k independent looks at a nominal 5% level, the chance of at least one false positive is 1 - 0.95^k: 26% at six segments, 64% at twenty. Quote the k you actually inspected, not the k you reported.
- Use the arithmetic already in front of you. On the point estimates, a +1.1pp mobile_web effect combined with a pooled +0.05pp implies the remaining surfaces average negative in proportion to mobile_web's share of exposures. State that as a testable implication of their story rather than as a rebuttal of it.
- Ask the one question that settles the category: was mobile_web named in the analysis plan before launch? If it was, it is a planned comparison and gets a corrected reading. If it was not, it is a hypothesis, and the honest move is to size the test that would confirm it.
- Convert the refusal into a cost. Size a mobile_web-only confirmatory test at the claimed effect, state the weeks of mobile_web traffic it needs, and close with the recommendation: do not ship this as a lift, and note that the interval already rules out anything at or above +0.6pp, which is itself a useful input to the roadmap.
Follow-up
- The confirmatory test you sized needs nine weeks of mobile_web traffic and the team has three. What do you recommend instead?
- Suppose mobile_web was pre-registered. How does your reading change, and what correction do you apply?
- Your interval excludes +0.6pp. Is that the same as saying the feature does nothing?
Turn an ambiguous onboarding question into a measurable metric
Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.
Approach
- Recognise what is being probed: whether you convert a goal into a computable predicate without stalling for requirements or guessing in silence. Listing clarifying questions is the generic answer; shipping a defensible default alongside them is the strong one, because the review is in two days and it will happen with or without you.
- Infer the decision behind the request. A question about whether onboarding works, arriving before a planning cycle, usually means whether to staff it next quarter. That points at a rate with visible headroom over several cohorts, not at a descriptive dashboard.
- Write the three questions so that each one changes the SQL. Which population, all signups or only self-serve from dim_user.signup_surface. What counts as working, reaching a core action or completing the onboarding flow_id. Against what bar, last quarter's cohorts or a stated target.
- Propose the default explicitly: seven-day activation on weekly signup cohorts. Numerator, users with is_core_action = TRUE events on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). Denominator, the signup cohort with is_internal = FALSE. Publish with an eight-day lag, and state that the two-distinct-days threshold is a frozen choice rather than a discovery.
- Name the exclusion in the same breath as the number. The series shows whether users activate; it does not establish that onboarding caused the level, which needs a staged rollout or an experiment.
Follow-up
- The director replies that they meant the onboarding flow specifically, not activation. What changes in the query and in the caveats?
- Your cohort metric needs an eight-day lag and the review is in two days. What do you present, and how do you label it?
- Two of your three questions come back unanswered. Which one do you refuse to proceed without?
- 01
You are writing the impact section of your own review. Over the year you ran four experiments, one of which shipped and three of which were flat; you corrected the definition of gross monthly revenue churn so that cancellation is recognised at period_end_utc; and you built a self-serve funnel dashboard. Weekly active accounts rose 14% over the same period. Your reviewer knows the data well. Write the three impact claims you would defend, stating for each what you contributed, what evidence supports it, and what portion of the outcome you are not claiming.
- 02
A feature you evaluated is flat on seven-day activation: +0.05pp with a 95% interval of [-0.47pp, +0.57pp], from 61,000 exposed users per arm in fct_experiment_exposure joined to dim_user and fct_event. Baseline activation is 32%. The launch team asks you to drop every surface except mobile_web, where the point estimate is +1.1pp, and re-run. You have ten minutes in their planning meeting. Deliver a spoken position: what you will and will not do, and the decision you recommend.
- 03
Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.
Are these confirmed Zomato interview questions?
No. Every prompt here is an original PracHub practice exercise written for the Data Scientist role and for product and business analytics data problems. This guide does not claim to reproduce Zomato's interview questions, rounds or hiring timeline. Confirm the actual format, team and scope with your recruiter.
PracHub Data Scientist practice ↗How can I practise A/B testing without an experimentation platform?
Simulate one. Write a script that draws two arms from known distributions, run your analysis on it, and confirm that a true null produces false positives at roughly your alpha and a known effect is detected at roughly your stated power. Then break it deliberately: peek early and stop on significance, add a correlated second metric, randomise by user but analyse by session. Failure modes you have caused yourself are the ones you can explain.
PracHub Data Scientist practice ↗How should I approach a take-home assignment?
Respect the stated time budget, then make your choices legible. Write down what you did not do and why you cut it. A clean, reproducible analysis that answers the question asked beats a sprawling notebook with six models. Put a short summary at the top for a hiring manager: the question, the answer, your confidence, the caveats. These are graded mostly on judgment and communication; the modelling rarely separates candidates.
PracHub Data Scientist practice ↗What if I think the interviewer's premise is wrong?
Test it before announcing it. Ask a clarifying question that separates the two readings, because the premise is often a deliberate simplification. If it really is wrong, say what you observed, why it changes the answer, and offer to proceed under either reading. Some cases plant a flawed premise on purpose and noticing it is the question being asked. Being right about it and graceless about it still costs you.
PracHub Data Scientist practice ↗Sources & methodology 2 sources ↗
Official role evidence, timestamped platform data and clearly labeled preparation advice.
- 01PracHub Data Scientist practice ↗
Cross-company practice questions for this role. Not an employer question record.
platform · Accessed 2026-09-22 - 02PracHub interview preparation framework ↗
The shared preparation framework these checkpoints and the seven-day plan follow.
platform · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Zuora Data Scientist Interview Questions & Guide 2026
Product-sense practice for a Zuora data scientist interview: define a product and business analytics metric, pick its guardrails, and argue why a tempting alternative would mislead you.
Zoox Data Scientist Interview Questions & Guide 2026
Case practice for the Zoox data scientist interview, where the prompt is deliberately underspecified and your first move is scoping it into something measurable.
ZoomInfo Data Scientist Interview Questions & Guide 2026
Product-sense practice for a ZoomInfo data scientist interview: define a product and business analytics metric, pick its guardrails, and argue why a tempting alternative would mislead you.
Zoom Communications Data Scientist Interview Questions & Guide 2026
SQL sets for data scientist prep ahead of Zoom Communications, covering window functions, funnel and retention queries, and the joins that quietly drop the rows you needed.