A Data Scientist at Nextroll sits at the intersection of machine learning, software engineering, and business strategy. Nextroll is a highly data-driven organization that powers programmatic advertising, retargeting, and audience segmentation across its primary business units, AdRoll and RollWorks. As a member of the data science team, you will design, build, and deploy algorithms that process billions of events daily, directly influencing real-time bidding (RTB) engines, budget pacing, fraud detection, and multi-touch attribution.
The impact of this role cannot be overstated. Your models will operate under strict latency constraints (often requiring decisions in milliseconds) and manage massive data volumes. This means that a Data Scientist at Nextroll is not just a researcher; you are an engineer of intelligent systems. You will work on optimizing click-through rate (CTR) predictions, building robust recommendation engines, and developing anomaly detection systems that protect advertisers from fraudulent traffic.
To succeed in this role, you must possess a strong foundation in statistical modeling and machine learning, coupled with the software engineering discipline required to write production-ready, highly optimized code. It is a challenging but highly rewarding environment where your work directly impacts company revenue and the advertising performance of thousands of global customers.
Recruiter Call
reportedA screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.
What to demonstrate
- Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
- Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
- Whether timeline, location and compensation expectations make the rest of the loop worth scheduling
How to prepare
- Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
- Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
- Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice
Technical Screen
reportedMuch of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.
What to demonstrate
- Whether the query you write matches the plan you just described
- What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
- Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly
How to prepare
- Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
- Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
- Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock
Onsite Interview
reportedA day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.
What to demonstrate
- Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
- Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
- Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method
How to prepare
- Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
- Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
- Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
- Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times
PracHub editorial advice for the preparation topics above.
Counting conversions without deduplicating across reporting sources and identity spaces.
The same purchase routinely arrives twice, once from a browser pixel and once from a server-side API, and the two reports carry different identifiers, so a naive count inflates conversions and deflates cost per action by the duplication rate. Deduplication depends on the advertiser populating a shared dedup_key correctly, which many do not, and the failure is silent because both rows look individually valid. The mirror-image error is under-counting: when a user clicks on a mobile app and converts in a desktop browser, the identity join fails and a real conversion is attributed to nothing, which makes the channel look worse than it is. Any conversion count reported without stating the dedup rule and the identity-match rate behind it should be treated as unverified.
Treating a bid as if it does not affect the price paid, under a first-price auction.
In a second-price auction the winner pays the runner-up's bid, so bidding true value is close to optimal and bid and price are usefully separable. Under first-price clearing, which is now the common case in open exchanges, the winner pays exactly what it bid, so raising the bid raises both the win probability and the cost of every win that would have been won anyway. A bid-increase experiment can therefore show more impressions, more conversions and worse profit simultaneously, and reading only the volume metrics will call it a success. The related second-order problem is that your own bid increase raises the clearing prices your control cells face, which contaminates the comparison unless the cells are separated by market.
SQL that silently fans out on a one-to-many join
State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.
Crediting a treatment for regression to the mean
Selecting a group because it is extreme (lowest-engagement users, accounts having their worst month, the bottom decile of a score) moves that group's expected next-period value back toward the average even under no treatment, by exactly as much as the selecting measure is imperfectly correlated with its own later value. Compare against units that met the same selection rule and went untreated, or use two pre-periods so the bounce-back is visible before the intervention starts. A pre-post number on a group chosen for being extreme measures the selection rule, not the treatment.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
How would you model the arrival rate of ad requests using a Poisson pr…
How would you model the arrival rate of ad requests using a Poisson process, and what are the limitations of this approach?
Approach
- Set a baseline first, so any model has something honest to beat.
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
How would you design a machine learning model to predict Click-Through…
How would you design a machine learning model to predict Click-Through Rate (CTR) for real-time bidding, and which evaluation metrics would you prioritize?
Approach
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- Where could label leakage enter this setup?
Simulate bid shading profit under first-price auction clearing
auctions holds 200,000 backtest rows for one line_item_id: predicted_cvr (per impression), floor_price_cpm_usd (nullable when the floor is undisclosed), and best_competing_cpm_usd, the highest competing bid, observable only in this sample. Clearing is first price. The advertiser values a conversion at $40. For shading factors s from 0.50 to 1.00 in steps of 0.05, the submitted bid is s * 1000 * 40 * predicted_cvr and the line item wins when that bid clears both the floor and the best competing bid. Return win rate, spend, conversions and profit per thousand auctions, the profit-maximising s, and an uncertainty estimate for it.
Approach
- Separate what is deterministic from what is random before writing any sampling code. Given the bid, winning is deterministic and the price paid under first-price clearing is exactly the submitted bid, so the only stochastic element is whether a won impression converts. Use the closed form sum of predicted_cvr over wins for the expected conversion count and reserve Monte Carlo for the profit distribution, not the mean.
- Vectorise across s: build the bid vector once, then evaluate the 11 shading factors as numpy operations over the full array. A per-row Python loop over 200,000 rows by 11 factors is about two million iterations and turns a 3-second job into minutes.
- Make the win condition explicit and handle nulls deliberately: treat a null floor as no floor, require bid > best_competing_cpm_usd, and require bid >= floor_price_cpm_usd. Ties at exactly the floor behave differently from ties against a competitor, and the convention should be stated rather than inherited from a comparison operator chosen by accident.
- Compute profit per thousand auctions as (40 * conversions - sum of bid/1000 over wins) / n * 1000. Spend must use your own submitted bid, not the competing bid: under first-price clearing every inframarginal win, one you would have taken at a lower bid, now costs more, and that term is what creates an interior optimum.
- Get uncertainty by bootstrapping auction rows, recomputing the whole curve per replicate, and reporting the distribution of the argmax s rather than only a confidence band around the profit level. The decision uses the argmax, and it is far less stable than the profit curve it comes from.
- State the backtest's limit honestly: best_competing_cpm_usd was observed while you bid what you actually bid. If your change alters competitors' behaviour or triggers exchange-side floor adjustments, the counterfactual does not hold and the curve overstates the achievable gain.
Worked solution 35 min
- Precompute value_cpm = 1000 * 40 * predicted_cvr and floor_eff = floor_price_cpm_usd filled with -inf.
- For each s: bid = s * value_cpm; win = (bid > best_competing_cpm_usd) & (bid >= floor_eff); spend = bid[win].sum()/1000; expected conversions = predicted_cvr[win].sum().
- Profit per thousand auctions = (40 * conversions - spend) / n * 1000; assemble the 11-row curve with win rate = win.mean().
- Bootstrap: resample row indices with replacement 500 times, recompute the curve on each replicate, and collect the argmax s plus a percentile band on profit at the point-estimate optimum.
- Optionally draw Bernoulli(predicted_cvr) on won rows for one replicate set to show the profit distribution around the expectation, and confirm the sampled mean matches the closed form.
Follow-up
- Raising this line item's bid also raises the clearing price faced by another line item in the same account bidding on overlapping supply. How does that change the optimum, and how would you detect it in the logs?
- predicted_cvr over-predicts by 15% at the head of the distribution. Which way does the optimal s move, and roughly how far?
- How would you run this as a live test instead of a backtest, and what is the randomisation unit given that both arms draw on one budget?
How would you merge and aggregate two large, mismatched datasets using…
How would you merge and aggregate two large, mismatched datasets using Python or SQL without overloading system memory?
Approach
- Say which table is the grain you start from, and join outward from it.
- State the window function and its partition and ordering out loud before writing it.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
Follow-up
- How would you verify this result without re-running the same query?
- What breaks if events arrive late or out of order?
Auction funnel stage counts and stage-to-stage drop-off
bid_request_log holds one row per (auction_id, line_item_id) evaluation, including line items that were considered and did not bid, with columns auction_id, request_ts, line_item_id, response_status (bid, no_bid, timeout, filtered), no_bid_reason and win_flag. ad_impression has impression_id, auction_id, line_item_id, served_ts, is_billable and ivt_status. ad_click has click_id, click_ts, impression_id (nullable), line_item_id (denormalised onto the click row when the click is written), ivt_status and is_billable. Return one row per line_item_id for a given UTC day with: auctions evaluated, auctions bid into, auctions won, impressions served, billable valid impressions, valid clicks, and the rate between each consecutive stage.
Approach
- State the grain of each table out loud before writing a join. bid_request_log is (auction_id, line_item_id); ad_impression is one row per impression; ad_click is one row per click event, carrying its own click_ts and a denormalised line_item_id plus a nullable impression_id, so one impression can produce several click rows and a click can be attributed to a line item without resolving its impression.
- Get the three auction-side stages from a single grouped scan of bid_request_log using COUNT(*) FILTER (WHERE ...). One pass, no self-joins, and the stages are guaranteed consistent with each other.
- Aggregate impressions and clicks in their own CTEs at their own grain, then LEFT JOIN the pre-aggregated results on line_item_id. Joining the raw tables together first multiplies rows at every step and the error compounds down the funnel.
- Decide what a click stage means and say so. COUNT(*) over ad_click counts click events; COUNT(DISTINCT impression_id) counts impressions that received at least one click, and silently drops the clicks whose impression_id is null. They are different numbers and the funnel reads differently under each.
- Say which clock the click stage is bucketed on. Filtering on click_ts places a click in the day it happened, which is not always the day its impression served: a click at 00:04 UTC on an impression from 23:58 the previous night lands in the next bucket, so a single-day CTR is approximate at the boundary. Bucketing clicks on the impression's served_ts instead removes that artefact but costs a join through impression_id and a decision about the null-impression_id clicks, which can then be counted in no bucket at all.
- Compute each rate against the stage immediately above it, with NULLIF on the denominator. The win-rate denominator is bids, not auctions evaluated.
Worked solution 20 min
- Write auctions AS (SELECT line_item_id, COUNT() AS evaluated, COUNT() FILTER (WHERE response_status = 'bid') AS bids, COUNT(*) FILTER (WHERE win_flag) AS wins FROM bid_request_log WHERE request_ts in the day GROUP BY 1).
- Write imp AS (SELECT line_item_id, COUNT() AS served, COUNT() FILTER (WHERE is_billable AND ivt_status = 'valid') AS clean FROM ad_impression WHERE served_ts in the day GROUP BY 1).
- Write clk AS (SELECT line_item_id, COUNT() AS clicks, COUNT() FILTER (WHERE impression_id IS NULL) AS unmatched FROM ad_click WHERE ivt_status = 'valid' AND is_billable AND click_ts in the day GROUP BY 1). This uses the click row's own line_item_id and click_ts, so the stage needs no join to ad_impression; the price of that is the midnight-boundary approximation above, which the unmatched count does not fix.
- LEFT JOIN the three CTEs on line_item_id and compute bids/evaluated, wins/bids, served/wins, clean/served and clicks/clean, each wrapped in NULLIF.
- Print the absolute counts beside the rates so a stage built on 40 rows is visible rather than hidden behind a percentage.
Follow-up
- This line item raised its bid, won more impressions, and its win rate fell. What mix shift produces that, and how do you show it from bid_request_log?
- Group the no-bid rows by no_bid_reason by hour. What shape tells you the budget exhausted early versus the targeting being too narrow all day?
- bid_cpm_usd, pre_shade_bid_cpm_usd and clearing_price_cpm_usd all sit on this table. What does the gap between them mean under first_price versus second_price clearing?
How would you approach building a bidding strategy for an advertiser w…
How would you approach building a bidding strategy for an advertiser with a fixed daily budget and a target Cost-Per-Acquisition (CPA)?
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- How would you detect that the metric is being gamed rather than genuinely improving?
Write a function to optimize a given data-processing pipeline, reducin…
Write a function to optimize a given data-processing pipeline, reducing its time complexity by several orders of magnitude.
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
An advertiser wants to know the incremental lift of their retargeting …
An advertiser wants to know the incremental lift of their retargeting campaign. How would you design an A/B test to measure this accurately?
Approach
- State the primary metric and the minimum effect worth shipping, then size the test.
- Name the randomisation unit first; it decides the variance and what the test can detect.
- Name the guardrails that would stop a launch even on a positive primary result.
Follow-up
- What would you conclude if the result is positive but the test is underpowered?
- How would you handle interference between treated and control units?
How would you design an anomaly detection system to identify sudden, u…
How would you design an anomaly detection system to identify sudden, unexpected spikes in ad traffic that could indicate bot activity?
Approach
- Clarify what is being asked and what a complete answer would contain.
- Say what you would check first and why it is the highest-information step.
- Work from the decision backwards to the evidence you would need.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Cut the variance of a market-level holdout with pre-period data
A 28-day holdout randomises 80 geo_dma markets 40/40; treated markets run the campaign and held-out markets get no delivery. The outcome is deduplicated conversion_event rows per market over the flight plus a 7-day window. Market sizes are heavily skewed and the unadjusted design detects only a 12% relative lift at 80% power. Twelve months of pre-period market conversions are available. Specify a variance-reduction plan, state the precision you expect, and state what would disqualify a covariate.
Approach
- Apply CUPED: define Y_adj = Y - theta*(X - mean(X)) where X is pre-period conversions for the same market and theta = Cov(Y,X)/Var(X), estimated on the pooled arms. Variance falls by a factor of (1 - rho^2), where rho is the correlation between pre-period and in-period outcomes.
- Estimate rho from history before the test runs, by regressing a past 28-day market window on its own preceding pre-period covariate. At rho = 0.9 the variance multiplier is 0.19, the standard error multiplier is 0.436, and the 12% MDE falls to roughly 5.2%.
- Disqualify any covariate measured at or after assignment. In-flight impressions, in-flight spend and in-flight clicks are all downstream of treatment; adjusting on them shifts the point estimate rather than only shrinking its variance.
- Stratify as well as adjust: form matched pairs on pre-period conversion volume, randomise within pair, and then analyse with pair fixed effects or a paired difference. Pairing that is not carried into the estimator buys nothing at all.
- Handle the skew at design time by analysing the ratio metric, conversions per thousand eligible bid requests, or by weighting on pre-period volume, and commit to the choice before seeing the arms.
- Pre-register theta's estimation rule and the covariate window, so neither can be tuned after the outcome becomes visible.
Worked solution 30 min
- On 12 months of history, build market-level 28-day outcome windows with matching pre-period covariates and compute rho; assume it returns 0.90.
- Compute the variance multiplier 1 - rho^2 = 0.19 and the standard error multiplier sqrt(0.19) = 0.436.
- Scale the unadjusted MDE: 12% * 0.436 = about 5.2% relative lift at 80% power, equivalent to 1/0.19 = 5.3 times the market count.
- Form 40 matched pairs on pre-period volume, randomise within pair, and write the estimator as a paired difference on CUPED-adjusted outcomes.
- Replay the whole pipeline as an A/A on two historical windows and confirm the adjusted estimate is centred on zero with the predicted standard error.
Follow-up
- CUPED moves the point estimate by 30% of its own standard error. What do you conclude?
- Would you use pre-period conversions or pre-period spend as the covariate, and why?
- How would the plan change if only 20 markets were available?
Attributed conversions vanished for six hours, then partly returned
Attributed conversions for 3 September show a hole between 02:00 and 08:00 UTC. A re-query the next morning shows most of the missing conversion_event rows now present, but attribution_credit for those touchpoint dates is still low and CPA on the dashboard is still wrong. conversion_event has conversion_ts and received_ts; attribution_credit is keyed on (conversion_id, touchpoint_id, attribution_model, model_version) and carries computed_at_ts. Determine what broke, what is still broken, and specify the repair.
Approach
- Count conversion_event by received_ts hour, not by conversion_ts. An ingestion outage is a hole on the arrival clock; on the event clock it looks like a dip that mysteriously heals, which is why the second query disagrees with the first.
- Confirm the backfill on the event clock: for the affected conversion_ts hours, compare the row count at the first query against the count now, and read the received_ts of the recovered rows to pin the recovery window.
- Ask whether anything downstream ran during the hole. Take max(computed_at_ts) per touchpoint date in attribution_credit. A computed_at_ts inside the outage means credit was computed against a partial conversion set and, if the job is not re-run, will never see the recovered rows.
- Distinguish the two failures in the writeup because they have different owners and different fixes: upstream ingestion lost six hours and self-healed, while the downstream attribution run is non-idempotent in effect and needs an explicit re-run for the affected touchpoint dates.
- Re-run attribution for those dates only, then verify the fix moved the affected dates and left neighbouring dates untouched before republishing.
Follow-up
- How would you make the attribution job safe to re-run, and what does it cost to recompute a touchpoint date every day until its window closes?
- What alert would have caught this in under 30 minutes, and on which clock would it be evaluated?
- Which already-published numbers need a restatement note, and what do you say in it?
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Practice prompt ↗Practice prompt ↗Worked solution ↗02Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Practice prompt ↗Practice prompt ↗03Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Practice prompt ↗Practice prompt ↗04The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Practice prompt ↗Practice prompt ↗06Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Practice prompt ↗Practice prompt ↗07Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.
How do you handle extreme class imbalance when training a model to det…
How do you handle extreme class imbalance when training a model to detect rare events, such as ad fraud?
Approach
- Close with what you would do differently, concretely.
- Name the disagreement or constraint, and how you resolved it with evidence.
- State the situation in two sentences and spend the rest on your reasoning.
Follow-up
- How did you know the outcome was caused by your change?
- What would you do differently if you ran that project again?
Correcting a published CPA win that was window maturation
Three weeks ago you published a note saying a bid-strategy migration cut attributed CPA by 18%. A colleague points out that you compared the first seven days after migration against a fully matured pre-period, while conversions for the recent dates were still arriving. Recomputed at equal maturity the improvement is 3%, inside noise. The note has been quoted in a quarterly business review deck. Deliverable: what you do in the next 24 hours, what you write and to whom, and the structural change that prevents a repeat.
Approach
- The interviewer is probing sequencing under embarrassment, so establish the corrected magnitude before writing anything: recompute both periods restricted to conversions received within the same number of days after conversion_ts, so a frozen window is compared to a frozen window.
- Build the account's maturation curve from received_ts minus conversion_ts, reporting the share of a click-day's eventual conversions landed by day 1, 3, 7 and 14. It both explains the error and becomes the artefact that prevents the next one.
- Enumerate the decisions taken since publication that cited the 18%: budget moved, line items paused, the deck slide, any advertiser-facing claim. Cost each one, because the correction's urgency is set by what was acted on, not by how wrong the number was.
- Send one message, not a drip, with the corrected number in the first sentence, the affected dates, the direction of the error, and an explicit statement of which decisions it does and does not reverse. Notify whoever presents the deck first.
- Ship the guard: a freeze rule in the query or dashboard so no period is reportable until click_window_days plus observed ingestion lag have elapsed, with immature dates badged rather than hidden.
Follow-up
- The corrected number had gone the other way and the migration looked better than you published. Does anything about your response change?
- How do you decide whether this warrants a written postmortem or a one-line correction?
Claiming impact for three projects with honest counterfactuals
You are writing a performance self-review. Over two quarters you shipped a bid-shading recalibration, a rewrite of the click-through-rate query that fixed an invalid-traffic filter applied to only one side of the ratio, and an incrementality test that led to a line item being paused. Deliverable: for each, the number you would claim, the counterfactual you claim it against, and the portion of credit that belongs to someone else. One of the three has no defensible dollar figure. Say which, and what you claim instead.
Approach
- The interviewer is probing whether you apply the same causal standard to your own work that you apply to an advertiser's, so write the counterfactual for each claim before the number: what would the metric have been over the same period without this work?
- For the shading recalibration, compare clearing price and profit on won impressions at matched supply mix across exchange_id, ad_format, device_type and geo_country, and report the profit change rather than the win-rate change; under first-price clearing a higher win rate can accompany worse profit.
- For the query fix, claim the magnitude of the bias removed rather than a revenue number: the gap between the old rate, which filtered invalid traffic on the numerator only, and the corrected symmetric rate, plus the decision that was being made on the wrong figure.
- For the incrementality test, claim the decision, not the savings. The paused budget's fate was never measured, so any dollar figure would assume that money vanished rather than moved. This is the item with no defensible number.
- Name specific collaborators and what they contributed, and pre-empt the obvious challenge by stating the confound you know about, such as seasonality or supply mix drift, and the bound you put on it.
Follow-up
- Your manager points out the paused line item's spend simply moved to another line item. Does your claimed impact survive?
- How do you claim credit for work whose value is a bad decision that did not happen?
- 01
How do you handle extreme class imbalance when training a model to detect rare events, such as ad fraud?
- 02
Three weeks ago you published a note saying a bid-strategy migration cut attributed CPA by 18%. A colleague points out that you compared the first seven days after migration against a fully matured pre-period, while conversions for the recent dates were still arriving. Recomputed at equal maturity the improvement is 3%, inside noise. The note has been quoted in a quarterly business review deck. Deliverable: what you do in the next 24 hours, what you write and to whom, and the structural change that prevents a repeat.
- 03
You are writing a performance self-review. Over two quarters you shipped a bid-shading recalibration, a rewrite of the click-through-rate query that fixed an invalid-traffic filter applied to only one side of the ratio, and an incrementality test that led to a line item being paused. Deliverable: for each, the number you would claim, the counterfactual you claim it against, and the portion of credit that belongs to someone else. One of the three has no defensible dollar figure. Say which, and what you claim instead.
Is this an official Nextroll interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Nextroll. Rounds and questions reflect what candidates have reported, not a process Nextroll has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How technical is the coding portion of the interview?
The coding interviews are highly technical and resemble software engineering screens. You are expected to write clean, optimized Python code, pay close attention to time and space complexity, and be prepared to optimize your code for speedups during the live session.
PracHub interview research ↗What is the primary focus of the case study round?
The case study round focuses on your ability to apply data science to real-world business problems. You will be evaluated on how you structure ambiguous questions, design experiments (like A/B tests), and align your technical solutions with key business metrics.
PracHub interview research ↗How does Nextroll view remote or hybrid work for this role?
Nextroll supports a flexible working model, offering hybrid and remote options depending on the team and location. It is best to clarify the specific expectations for your target team with your recruiter during the initial call.
PracHub interview research ↗What is the typical timeline from the first screen to an offer?
The process generally takes between 3 to 5 weeks. However, candidates have noted that Nextroll can be highly accommodating and accelerate the timeline significantly if you have competing offers.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
tvScientific Data Scientist Interview Questions & Guide 2026
Timed practice for tvScientific data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
Truliant Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for Truliant data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.
StackAdapt Data Scientist Interview Questions & Guide 2026
Experiment-design practice for StackAdapt data scientist candidates, covering randomisation units, power, minimum detectable effect and the traps that invalidate a result.
Taboola Data Scientist Interview Questions & Guide 2026
Take-home style practice for data scientists preparing for Taboola: a messy dataset, a fixed time budget, and a written analysis, reviewed for whether it states its assumptions and whether a reader could reproduce your numbers.