As a Data Scientist at QualityKiosk, you operate at the intersection of performance engineering, quality assurance, and advanced analytics. QualityKiosk is a global leader in digital quality engineering, and the Data Scientist role is pivotal in transforming massive amounts of testing, performance, and user-experience data into actionable insights for global enterprises. You will be responsible for building predictive models, optimizing testing frameworks, and ensuring that the digital journeys of end-users are seamless and robust.
This position demands a unique blend of technical rigor and product intuition. You will work closely with engineering and client-facing teams to diagnose performance bottlenecks and design experiments that improve system reliability. Whether you are analyzing traffic patterns or building anomaly detection models, your work directly influences the stability and success of complex, high-scale digital platforms. It is a role for those who enjoy solving high-stakes problems where data precision is paramount to business success.
Initial Screening
reportedMost candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.
What to demonstrate
- Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
- Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
- The substance of the questions you ask back, which an experienced screener reads as a level signal
How to prepare
- Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
- Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
- Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered
Panel Discussions
reportedWhere a loop ends with a senior leader, that conversation is rarely another skills test. The technical signal already exists by then, so the questions tend to open up: what you would look at first, where a metric you have heard about could mislead, what you would push back on. The decision being made is scope, which in practice means level and how much you would be trusted to own unsupervised. Treating it as a formality is the usual mistake. An open question late in the day is still being scored, and a vague answer reads as someone who has not run anything themselves.
What to demonstrate
- Whether your view of the business has anything specific behind it, given that you are working only from what is public and are expected to say so
- Whether the scope of work you describe owning matches the scope of the role, instead of sitting a level below it
- Whether you can disagree with something concrete and stay useful about it, rather than agreeing with everything said in the room
- Whether your questions are ones only this person could answer, as opposed to ones the recruiter already covered
How to prepare
- Build one view you could defend for two minutes using only public information: what the funnel probably looks like, which metric likely drives decisions, and where that metric could mislead. Being wrong for a stated reason survives this round; having no view does not
- Write down the largest piece of work you have owned from question to decision, who else touched it, and what you decided alone, then check that it reads at the level you are interviewing for
- Prepare one thing you would want changed if you joined and phrase it as a question rather than a verdict, so it opens a conversation instead of closing one
Live Technical Challenges
reportedThis round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.
What to demonstrate
- Whether your row counts survive each join, and whether you notice on your own when they do not
- Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
- Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
- Reaching a defensible answer inside the window instead of a refined one after it
How to prepare
- Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
- Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
- Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither
Behavioral Discussions
reportedMost of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.
What to demonstrate
- Whether you can state the other side's argument accurately before you explain why you disagreed
- What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
- Whether you distinguish being overruled from being wrong, and can give an example of each
How to prepare
- Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
- For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
- Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.
Final Assessment
reportedWhere a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.
What to demonstrate
- Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
- Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
- Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
- Whether you ask what decision is actually on the table before explaining anything
How to prepare
- Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
- For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
- Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else
PracHub editorial advice for the preparation topics above.
Slicing a flat experiment until a segment reaches significance
Testing one metric across twenty segments at a nominal 5% level produces a significant result about two thirds of the time when nothing is happening anywhere, and the segment that surfaces is by construction the one with the most favourable noise. The reported effect in that slice is then badly overstated, because selection on significance conditions the estimate on being large. What makes it dangerous rather than merely wrong is that a post-hoc segment always has a plausible story attached, so it survives the meeting. The controls are declaring the small number of segments of interest before launch, correcting across the ones tested, and treating anything discovered afterwards as a hypothesis that needs its own adequately-powered test rather than a finding.
Watching an experiment daily and stopping when it crosses significance
A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.
Building features from data that postdates the prediction time
Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.
Extrapolating a first-week lift inflated by novelty effects
Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Explain the concept of Type I and Type II errors and how they impact y…
Explain the concept of Type I and Type II errors and how they impact your decision-making.
Approach
- Write down the assumption the method needs before you use the method.
- Sanity-check the answer against a simple bound or a simulated case.
- Quantify uncertainty explicitly rather than reporting a point estimate alone.
Follow-up
- Which assumption here is most likely to be violated in practice?
- What sample size would you need to detect an effect half this size?
Split a pooled conversion drop into rate and mix
You have weekly visit-to-signup counts by segment: a DataFrame with week, device_type, referrer_channel, visitors and signups. The pooled rate fell 0.84 percentage points between two consecutive weeks while several individual segments rose. Write a function that, for a caller-supplied list of segment columns, splits the pooled change into a rate effect, a mix effect and an interaction term that sum exactly to the observed change. Return those three scalars plus a per-segment contribution table sorted by absolute contribution, so the largest single driver can be named.
Approach
- State the algebra before coding: the pooled rate is r = sum over segments of w_s * r_s, with w_s the segment's share of the denominator. Then r1 - r0 decomposes exactly into sum(w_s0 * (r_s1 - r_s0)) for rate, sum((w_s1 - w_s0) * r_s0) for mix, and sum((w_s1 - w_s0) * (r_s1 - r_s0)) for interaction. The identity is per-segment, so it holds for any numbers you put in the four slots.
- Pivot both weeks onto a common segment index with an outer join so a segment that appeared or vanished is kept rather than dropped, then decide what rate to give a segment with no visitors in one of the weeks, and document the choice. The identity stays exact either way because the missing week's weight is 0, but the attribution does not. Filling the missing rate with 0 sends an appearing segment's entire w_s1 * r_s1 into the interaction term, since w_s0 = 0 makes both the rate term and the mix term (w_s1 - w_s0) * r_s0 identically zero; a vanishing segment then splits as -w_s0 * r_s0 in rate, -w_s0 * r_s0 in mix and +w_s0 * r_s0 in interaction.
- The convention used below instead imputes the missing week's rate as that week's pooled rate. A vanishing segment then lands wholly in mix at -w_s0 * r_s0, with rate and interaction cancelling; an appearing segment puts w_s1 * r_pooled0 in mix (volume arriving at the average rate) and only w_s1 * (r_s1 - r_pooled0) in interaction (its rate differing from that average). Impute by which week the segment is missing from, never by argument order, or the swap identities below stop holding.
- Guard the division where visitors is 0 so no NaN enters the vectors, because a single NaN poisons every sum. A segment with zero visitors in both weeks contributes exactly 0 and can be dropped; a segment missing from only one week does not contribute 0, and where its contribution lands is settled by the convention above, not by the guard.
- Compute the three components as vectors over segments, then sum. Keep the vectors, because the per-segment contribution table is what turns the decomposition into an explanation.
- Assert that the three components sum to the observed pooled change within floating-point tolerance. This identity is exact, so a mismatch means an implementation bug, not a modelling judgement.
Follow-up
- The mix effect accounts for 0.71 of the 0.84 point drop, driven by paid_social volume. What is your recommendation, and what would change it?
- Why is a two-way split into a counterfactual rate and a residual also exact, and when would you prefer it to the three-way version?
- Segmenting on device and channel leaves a large interaction term. What does that tell you about the choice of segments?
Simulate the false positive cost of repeated peeking
Quantify the cost of peeking. Simulate a two-arm experiment with no true effect: each arm accumulates Bernoulli conversions at a base rate of 0.10 up to 40,000 units per arm. Run a two-sided two-proportion z-test at alpha 0.05 at ten equally spaced interim points, and record whether the test ever crossed. Report the false positive rate over at least 10,000 replications, alongside the rate for a single look at the final sample only. Use a fixed seed and report a Monte Carlo standard error on both figures.
Approach
- Generate each replication as two cumulative sums of Bernoulli draws, then read the interim points off the cumulative arrays. Regenerating data at each look would make the looks independent, which destroys exactly the dependence the exercise is about: later looks share data with earlier ones.
- Use the pooled-variance two-proportion z: p_pool = (x1+x2)/(n1+n2), z = (p1-p2) / sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2)), reject when |z| > 1.96. State that the normal approximation is fine here because the smallest look has roughly 400 expected conversions per arm.
- Vectorise across replications rather than looping: draw a (reps, n) array of uniforms, threshold at 0.10, cumsum along axis 1 and slice the ten look indices. A per-replication loop at 10,000 by 40,000 is unnecessarily slow.
- Record the any-cross indicator per replication, take the mean, and compute the Monte Carlo standard error as sqrt(p*(1-p)/reps) so the reported figure comes with its own precision.
- Report the single-look rate in the same run as a control. If it does not land near 0.05, the bug is in the test statistic and not in the peeking argument.
Worked solution 30 min
- rng = np.random.default_rng(seed); for memory, batch the replications in chunks and accumulate the any-cross count across chunks.
- Per chunk: draw (chunk, 40000) uniforms per arm, x = (u < 0.10).cumsum(axis=1), slice columns at indices 3999, 7999, ..., 39999.
- Compute the ten z statistics vectorised over the chunk, take crossed = (np.abs(z) > 1.96).any(axis=1).
- Aggregate: peek_rate = total_crossed / reps; single_rate = mean of |z_final| > 1.96; mc_se = sqrt(p*(1-p)/reps) for each.
Follow-up
- Re-run with 40 looks instead of 10. Why does the curve flatten rather than continue rising linearly?
- Among the replications that crossed, what is the mean observed lift, and why is it not zero?
- What does an O'Brien-Fleming boundary or an always-valid confidence sequence change about this simulation, and what does each cost in power?
How would you handle missing or noisy data in a production performance…
How would you handle missing or noisy data in a production performance dataset?
Approach
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- State the window function and its partition and ordering out loud before writing it.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
Follow-up
- How would you verify this result without re-running the same query?
- How does the query change if the join becomes one-to-many?
Explain the difference between inner, left, and full joins when dealin…
Explain the difference between inner, left, and full joins when dealing with massive, sparse datasets.
Approach
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- Say which table is the grain you start from, and join outward from it.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
Follow-up
- What breaks if events arrive late or out of order?
- How would you verify this result without re-running the same query?
Write a query using SQL window functions to calculate a rolling averag…
Write a query using SQL window functions to calculate a rolling average of system latency over the last 30 days.
Approach
- State the window function and its partition and ordering out loud before writing it.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Follow-up
- How does the query change if the join becomes one-to-many?
- How would you verify this result without re-running the same query?
Pair flow starts to completions within a thirty-minute bound
fct_event holds event_id, flow_id, flow_instance_id, event_name, occurred_at_utc, surface, app_version. Using only rows where event_name is 'flow_started', 'flow_completed' or 'error_shown', compute the daily core-flow completion rate by surface and app_version. Numerator: distinct flow_instance_id whose flow_completed occurs within 30 minutes of its flow_started with no error_shown for the same instance in between. Denominator: distinct flow_instance_id with a flow_started that day. Report rows with a NULL flow_instance_id as a separate coverage count, not inside either side.
Approach
- Collapse the stream to one row per flow_instance_id using conditional aggregation: MIN(occurred_at_utc) FILTER (WHERE event_name = 'flow_started') AS started, MIN(...) FILTER (WHERE event_name = 'flow_completed') AS completed, MIN(...) FILTER (WHERE event_name = 'error_shown') AS first_error. This makes the 'first' semantics explicit and avoids a self-join entirely.
- Apply the predicates on that single row: completed IS NOT NULL AND completed <= started + interval '30 minutes' AND (first_error IS NULL OR first_error > completed). Note the last clause encodes 'in between' literally, so an error surfaced after a successful completion does not disqualify the attempt.
- Bucket by started::date, surface and app_version. Splitting by app_version is not decoration: a completion-rate regression is nearly always confined to one client build, and a pooled daily rate hides it behind the installed base.
- Count NULL flow_instance_id rows as a separate coverage figure. They cannot be attributed to an attempt at all, so they belong in neither numerator nor denominator, and their share tells you how much of the rate is unmeasurable.
- For any roll-up to week or to all surfaces, re-sum the numerator and denominator rather than averaging the daily rates.
Worked solution 25 min
- Write the per-instance collapse CTE and confirm COUNT(*) equals COUNT(DISTINCT flow_instance_id).
- Add the three predicates one at a time, recording how many instances each removes.
- Aggregate by day, surface and app_version with FILTER on the clean-completion flag.
- Compute the NULL-instance coverage count as a separate select and report it alongside.
- Pick the worst day-surface-version cell and pull ten raw instances to confirm the disqualification reason.
Follow-up
- occurred_at_utc is client-supplied. What do you do with an instance whose completion timestamp precedes its start?
- flow_instance_id is missing on 4 percent of starts for one app_version. What can and cannot you conclude about that build?
- An instance starts at 23:55 and completes at 00:04. Which day owns it, and what does the alternative do to the daily series?
How would you design a metric to measure the success of a new performa…
How would you design a metric to measure the success of a new performance monitoring feature?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
If a key business metric suddenly drops by 20%, what is your step-by-s…
If a key business metric suddenly drops by 20%, what is your step-by-step process for diagnosing the root cause?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How do you determine if a result is statistically significant when you…
How do you determine if a result is statistically significant when your sample size is limited?
Approach
- State the primary metric and the minimum effect worth shipping, then size the test.
- Name the randomisation unit first; it decides the variance and what the test can detect.
- Decide the analysis before seeing data, including how long it runs and when you look.
Follow-up
- How would you handle interference between treated and control units?
- What would you conclude if the result is positive but the test is underpowered?
How do you define and calculate statistical significance in the contex…
How do you define and calculate statistical significance in the context of high-frequency performance data?
Approach
- Decide the analysis before seeing data, including how long it runs and when you look.
- Say whether units interfere with each other, and switch design if they do.
- Name the guardrails that would stop a launch even on a positive primary result.
Follow-up
- What would you do if you could not randomise at all?
- How would you handle interference between treated and control units?
How do you balance the need for high system performance with the cost …
How do you balance the need for high system performance with the cost of infrastructure?
Approach
- Say what you would check first and why it is the highest-information step.
- State your assumptions explicitly before working the problem.
- Work from the decision backwards to the evidence you would need.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Diagnose a 50.5 to 49.5 split before reading the result
An experiment randomised on visitor_id ran for 11 days. fct_experiment_exposure holds 104,320 control rows and 102,180 treatment rows for the experiment_id, all with is_in_analysis_population = TRUE, against a configured 50/50 split. The treatment arm shows a 3.1% relative lift in visit-to-signup conversion at p = 0.004, and the team wants to ship. Decide whether that lift can be reported, show the arithmetic behind your decision, and name the three checks you would run first on the exposure logging.
Approach
- Test the split instead of eyeballing it. Chi-square goodness of fit against 103,250 expected per arm gives 2 x 1070^2 / 103,250 = 22.2 on 1 degree of freedom, p about 2.5e-6. The conventional alarm threshold is p < 0.001, so this is a sample ratio mismatch.
- Treat the mismatch as invalidating, not as a caveat. The arms are no longer exchangeable, so the 3.1% lift has no causal reading and rebalancing on covariates afterwards does not restore it.
- Walk the logging path in the order units flow through it: assignment service, then the exposure event fired on the treated surface, then the filters that build the analysis population. Each stage can drop one arm preferentially.
- Name the three suspects this stack actually has. Exposure emitted by client code the treatment arm reaches later or not at all when its bundle errors; an extra redirect or render in one arm that loses slow connections; and a post-assignment filter (is_bot_flagged, is_internal, consent_state) that correlates with variant.
- Check whether the imbalance is uniform by splitting counts on session_date, surface, app_version and device_type. A mismatch confined to one client build or to day one points at a deployment fault that can be fixed and re-run, not at a design fault.
Worked solution 15 min
- N = 206,500, expected 103,250 per arm, observed deviation 1,070, which is 0.518% of the total pushed into one arm.
- Chi-square = 2 x 1070^2 / 103,250 = 22.18 on 1 df, p about 2.5e-6.
- Compare with the p < 0.001 threshold and stop: the readout is void.
- Re-pull the same counts grouped by session_date and app_version to localise where the rows were lost.
- Deliver one line: the split is off by 1,070 visitors, a roughly one-in-400,000 event under correct randomisation, so the experiment is being fixed and re-run rather than read.
Follow-up
- The imbalance is entirely in one app_version on android. Can you salvage the other surfaces, and under what argument?
- How would you monitor this across every running experiment without drowning in false alarms from a daily chi-square at alpha 0.05?
- The missing rows turn out to be disproportionately visitors with consent_state = 'unknown'. What does that do to the estimate even after the logging bug is fixed?
Attribute a flow-completion drop to one client build
Core-flow completion rate on surface = 'ios', defined as distinct flow_instance_id reaching 'flow_completed' within 30 minutes of 'flow_started' with no 'error_shown' on the same id in between, fell from 78% to 71% over four days. A new iOS build began a staged rollout on day one. Using fct_event columns flow_id, flow_instance_id, app_version, event_name, user_id and occurred_at_utc, produce the completion rate per build per day, size the loss in absolute completions, and say whether the fix is a rollback. Treat the possibility that the instrumentation changed rather than the flow.
Approach
- Cut the rate by app_version and day, each build against its own denominator. A pooled series during a staged rollout is a weighted average whose weights move daily, so it declines in proportion to the rollout share even when the old build is perfectly flat; confirming that proportionality is itself the evidence the regression is build-specific rather than environmental.
- Test the denominator before believing the rate. Compute flow_started events per distinct user_id on each build: if the new build re-mints flow_instance_id on retry, or fails to carry the id from start to completion, the denominator inflates and the numerator falls with nothing changing for the user.
- Normalise to a user-level outcome that is immune to the id question, namely completions per distinct user_id per day on each build. This is the number that says whether anyone actually failed to finish.
- Check the error path: pull error_shown counts and their properties for flow_instance_id values on the new build. A genuine regression produces errors; an instrumentation break produces missing completions with no corresponding error volume, and the two prescribe different fixes.
- Size the real loss as the per-user completion gap multiplied by daily users on the new build, and give the rollback recommendation conditional on the error evidence rather than on the rate.
Follow-up
- The rollout is at 45% and product wants to go to 100% tomorrow. What do you say, and what would you need by when?
- If the flow_instance_id is genuinely being re-minted, what is the correct historical treatment of the four affected days?
- How would you have caught this on day one instead of day four?
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Practice prompt ↗Practice prompt ↗Practice prompt ↗03Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Practice prompt ↗Practice prompt ↗Practice prompt ↗04The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Practice prompt ↗Practice prompt ↗06Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Practice prompt ↗Practice prompt ↗07Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.
Give an example of a project where you had to pivot your approach due …
Give an example of a project where you had to pivot your approach due to changing requirements or data limitations.
Approach
- Quantify the outcome, including what you would not claim credit for.
- Pick a story where you drove the decision, not one where you observed it.
- State the situation in two sentences and spend the rest on your reasoning.
Follow-up
- What would you do differently if you ran that project again?
- What did you decide not to do, and why?
Disagree with a product manager's roadmap claim using data
A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.
Approach
- Recognise what is being probed: whether you can separate a number being right from an inference being wrong, and do it without costing the PM face. The generic answer recites that correlation is not causation; the strong one names the specific confound and proposes the cheapest design that could distinguish the explanations.
- State the alternative concretely. Accounts that connect an integration in week one are accounts that already have a workflow and a technical owner, so week-one intent plausibly drives both the connection and week-four retention. The selection is on intent, which no amount of post-hoc adjustment observes.
- Order the discriminating analyses by cost. First, condition on pre-connection activity by comparing retention within strata of week-one core-action count, which removes the crude version of the confound but not unobserved intent. Second, look for variation in integration availability that was unrelated to intent, such as a staggered release or an outage window. Third, an encouragement design that randomises a prompt to connect and reads the intent-to-treat effect on week-four retention, which is the only version that identifies an effect.
- Run the timing check, because it is nearly free and it is the most persuasive single piece of evidence. If the retention advantage among connectors is already visible before any of them connected, the causal story is largely finished.
- Pre-commit to what would change your mind and say it before you show anything: if the gap survives stratification and the encouragement arm moves week-four retention at all, the feature has a case and you will say so.
- Open the 1:1 by agreeing with the true part, that the correlation is real and worth chasing, then ask what effect size the roadmap plan assumes. That makes the size of the claim the topic instead of its authorship.
Follow-up
- The encouragement test needs six weeks and the roadmap locks in two. What do you recommend in the interim?
- Stratifying on week-one activity closes half the gap. What do you conclude, and what do you still not know?
- How would you word this in the roadmap document so the PM's original number is reframed rather than deleted?
Explain a wide interval to a non-technical executive
A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.
Approach
- Recognise what is being probed: whether you can carry uncertainty into a decision instead of either hiding it or hiding behind it. The generic answer promises to explain the interval in plain English; the strong one replaces the question 'what is the number' with 'across this range, where does the decision change'.
- Find the threshold before you draft anything. Ask what the pricing case assumes, then compute the conversion drop at which the higher price stops adding revenue: price uplift on the conversions kept against the revenue lost from conversions forgone. That single figure is what makes the range legible.
- Restate the estimate and both bounds in the unit the audience already reasons in. Convert percentage points into monthly first-paid conversions at current trial volume, then into mrr_cents_constant_fx, so the slide reads as money per month rather than as statistics.
- Place the range against the break-even and say which part of it sits on each side. If most of the range clears the threshold, that is a recommendation to proceed with a monitoring plan; if the range straddles it, that is a recommendation to narrow the range first.
- Name what would narrow it and what that costs in weeks, then give one recommendation with an explicit condition for revisiting it. Uncertainty stated without a next step is read as indecision and the midpoint gets used anyway.
Follow-up
- The executive says to give the midpoint and they will manage the risk. What do you do?
- How does the slide change if the interval were a 4.6pp to 0.2pp drop, with no positive outcomes in range?
- Why is a geo holdout the credible read here rather than the attributed channel numbers you already have?
- 01
Give an example of a project where you had to pivot your approach due to changing requirements or data limitations.
- 02
A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.
- 03
A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.
Is this an official QualityKiosk interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at QualityKiosk. Rounds and questions reflect what candidates have reported, not a process QualityKiosk has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult are the technical assessments?
The technical rounds are designed to test your real-world application of skills. If you are comfortable with SQL, experimental design, and basic statistics, you will be well-positioned to succeed.
PracHub interview research ↗What differentiates successful candidates?
Beyond technical skill, successful candidates are those who demonstrate clear, structured thinking. Candidates report that interviewers value clarifying questions asked before jumping into a solution.
PracHub interview research ↗Is there a focus on machine learning?
While machine learning is a component of QualityKiosk's work, the primary focus for this role is on diagnostics, experimentation, and performance analytics.
PracHub interview research ↗What is the typical timeline?
The process is designed to be quick. Candidates report that feedback usually arrives promptly after each stage.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Fetch Rewards Data Scientist Interview Questions & Guide 2026
Product-sense practice for a Fetch Rewards data scientist interview: define a product and business analytics metric, pick its guardrails, and argue why a tempting alternative would mislead you.
1 digit technology Pvt Data Scientist Interview Questions & Guide 2026
Take-home style practice for data scientists preparing for 1 digit technology Pvt: a messy dataset, a fixed time budget, and a written analysis, reviewed for whether it states its assumptions and whether a reader could reproduce your number
xAI Data Scientist Interview Questions & Guide 2026
Communication practice for xAI data scientist candidates: turn an analysis into a recommendation, name the uncertainty honestly, and answer the follow-up a skeptical stakeholder will raise.
Windfall Data Scientist Interview Questions & Guide 2026
Data-quality drills for Windfall data scientist prep: survivorship and selection bias, leakage through time-ordered features, and extreme values where provenance decides the call. A logging fault or unit slip gets corrected or dropped; a ge