A Data Scientist at OpenX works at the absolute frontier of programmatic advertising and high-throughput technology. OpenX operates one of the world's largest independent advertising exchanges, processing billions of transactions daily and generating massive volumes of real-time data. In this role, you are not just analyzing static datasets; you are building the intelligent algorithms that power real-time bidding, yield optimization, ad fraud detection, and traffic shaping.
The impact of your work is immediate and highly visible. A minor optimization in a machine learning model can lead to significant revenue shifts for publishers and advertisers alike. Your primary challenge will be to balance statistical rigor with computational efficiency, ensuring that complex predictive models can execute within milliseconds to keep pace with the lightning-fast programmatic ecosystem.
To succeed as a Data Scientist or Staff Data Scientist at OpenX, you must possess a rare combination of deep mathematical intuition, strong software engineering fundamentals, and a product-oriented mindset. You will collaborate closely with platform engineers, product managers, and business leaders to turn massive-scale data into actionable marketplace intelligence.
Recruiter Screen
reportedData Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.
What to demonstrate
- Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
- Whether you name what you have not done instead of stretching to cover every line of the posting
- Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage
How to prepare
- Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
- Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
- Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live
Technical Conversation
reportedThis round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.
What to demonstrate
- Whether your row counts survive each join, and whether you notice on your own when they do not
- Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
- Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
- Reaching a defensible answer inside the window instead of a refined one after it
How to prepare
- Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
- Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
- Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither
Hands-on Coding Assessment
reportedBefore anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.
What to demonstrate
- Whether an ambiguous term becomes a specific column and filter before any computation happens
- Whether you read the schema for keys and cardinality rather than only for column names
- Whether the result answers the question at the grain it was asked at, per user or per session or per day
How to prepare
- Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
- On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
- Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over
PracHub editorial advice for the preparation topics above.
Randomising users into treatment and control while both arms draw from the same campaign budget.
The arms compete in the same auctions and against the same budget, so treatment winning more impressions directly starves control, and the measured gap includes that cannibalisation rather than only the ad effect. The test looks methodologically clean, the randomisation is genuinely valid, and the lift is still partly manufactured. This is an interference violation, not a randomisation failure, so checking balance on covariates will not catch it. The fixes are to randomise at a unit that contains the budget, such as a geographic market, or to give each arm its own budget and its own pacing, and then be explicit that you are now comparing two separately-budgeted campaigns.
Reallocating budget using last-click attribution.
Last-click assigns full credit to whichever touchpoint sits closest to the conversion in time, which structurally favours channels that harvest existing demand — retargeting an already-interested user, or catching a branded search — over channels that create demand in the first place. Optimising against it therefore moves money toward tactics that would have converted many of those users anyway, and the reported cost per action improves at the exact moment true incremental performance gets worse. The signature of this failure is a portfolio where every channel's attributed conversions sum to well above the advertiser's total conversion count. The counter is to treat attributed numbers as a budget-splitting convention and to source the actual reallocation decision from holdout-based incrementality.
Sizing estimates built on unnamed, unrevisable assumptions
Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.
Generalising beyond the population the sample actually supports
State the frame the sample was drawn from and where it diverges from the population you want to talk about: time window, platform, geography, opt-in. If a group is excluded from the frame, either weight to known margins or narrow the claim rather than quietly extending it.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Walk me through the bias-variance tradeoff. How does increasing model …
Walk me through the bias-variance tradeoff. How does increasing model complexity affect both bias and variance?
Approach
- Quantify uncertainty explicitly rather than reporting a point estimate alone.
- Write down the assumption the method needs before you use the method.
- Say what the estimate is of, and over what population it generalises.
Follow-up
- How would you explain this result to someone who does not know statistics?
- What sample size would you need to detect an effect half this size?
Describe a time when you had to explain a highly complex machine learn…
Describe a time when you had to explain a highly complex machine learning model to a non-technical product manager. How did you structure your communication?
Approach
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- Where could label leakage enter this setup?
- How would you choose the decision threshold, and who owns that choice?
How do you handle highly imbalanced datasets when training a binary cl…
How do you handle highly imbalanced datasets when training a binary classification model for ad click prediction?
Approach
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
Sessionise a device event stream with a 30-minute gap
events holds about 5 million unsorted rows: device_id (nullable, roughly 18% null), event_ts, event_type in {impression, click, landing_arrival, conversion}, line_item_id, value_usd (nullable). Define a session as a maximal run of events for one device_id in which consecutive events are at most 30 minutes apart. Assign a session_id to every sessionisable row and return a per-session summary with device_id, start_ts, end_ts, n_impressions, n_clicks, converted and total value_usd. Do not use a library sessionisation helper, and do not scan the frame more than once after sorting.
Approach
- Separate null device_id rows first and report their share. Null identity is a supply-source property, not random missingness, so grouping them under one null key would fabricate a single session spanning millions of unrelated events; excluding them means the summary describes identified traffic only, and that has to be said out loud.
- Sort once by (device_id, event_ts) with a deterministic tiebreak such as an event_type rank then a row id. Without a tiebreak, an impression and its click sharing a timestamp order differently between runs and the summary is not reproducible.
- Compute the boundary mask as (device_id != device_id.shift()) | (event_ts - event_ts.shift() > 30min) and take its cumsum as session_id. This is a grouped run-length in two vectorised operations: O(n log n) for the sort and O(n) afterwards.
- Create indicator columns (is_impression, is_click, is_conversion) before the groupby, then summarise with one named aggregation pass. Building each count from its own filtered groupby re-scans the frame per metric and violates the single-pass constraint.
- Assert the structural invariant rather than eyeballing it: within a device, sessions ordered by start_ts must be disjoint and increasing, every within-session consecutive gap at most 30 minutes, every between-session gap strictly greater.
Worked solution 25 min
- Split on device_id.notna(), record the null share, and keep the null rows aside rather than dropping them silently.
- Add an event_type rank column, sort by (device_id, event_ts, type_rank, row_id), and reset the index.
- boundary = (device_id != device_id.shift()) | (event_ts.diff() > pd.Timedelta('30min')); session_id = boundary.cumsum().
- Add is_impression/is_click/is_conversion indicators, then one groupby('session_id').agg giving device_id first, start_ts min, end_ts max, the three sums, converted as is_conversion.max() > 0, and value_usd sum.
- Run the invariant assertions per device on the summary before returning it.
Follow-up
- A conversion arrives three hours after the click and lands in its own session. Is that the right answer, and how does a session boundary differ from an attribution window?
- One device shows negative gaps from a clock four hours off. What does your boundary mask do with it, and what would you prefer it did?
- Make the 30 minutes a parameter. How would you choose it from the data rather than from convention?
Given an array of integers, find the contiguous subarray which has the…
Given an array of integers, find the contiguous subarray which has the largest sum and return its sum.
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Follow-up
- How does the query change if the join becomes one-to-many?
- What breaks if events arrive late or out of order?
Write a function to identify duplicate ad request IDs within a rolling…
Write a function to identify duplicate ad request IDs within a rolling time window using a live coding environment.
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Say which table is the grain you start from, and join outward from it.
- State the window function and its partition and ordering out loud before writing it.
Follow-up
- How would you verify this result without re-running the same query?
- What breaks if events arrive late or out of order?
Attributed cost per action without double counting spend or credit
attribution_credit holds one row per (conversion_id, touchpoint_id, attribution_model, model_version), with touchpoint_ts, line_item_id, credit_fraction, credited_value_usd and click_window_days, the model's click lookback in whole days, which is constant within one (attribution_model, model_version) pair; credit_fraction sums to 1.0 across a conversion's touchpoints within one model and one version. ad_impression has line_item_id, served_ts, billed_price_micros_usd and is_billable. For a UTC date range, return per line_item_id and day: spend, attributed conversions, attributed CPA, attributed ROAS, and a maturity flag derived from the model's own click window, for one named attribution_model and model_version. Both sides must land in the same daily buckets. State which timestamp each side is bucketed on and why.
Approach
- Pin attribution_model AND model_version before aggregating anything. The table stores several models, and several versions of each, for the same conversion. An unfiltered SUM(credit_fraction) therefore reports conversions multiplied by the number of (model, version) pairs present, which is usually a clean integer multiple and so survives a sanity glance.
- Bucket credit on touchpoint_ts, never on conversion_ts. Spend is incurred when the impression served, so aligning the outcome to the touchpoint puts numerator and denominator on the same cohort. Bucketing on conversion_ts shifts every conversion forward by a variable lag and produces day-over-day CPA swings that are pure bucketing artefacts and cannot be corrected downstream.
- Aggregate the two sides in separate CTEs at their own grain, then FULL OUTER JOIN on (line_item_id, day). Joining ad_impression to attribution_credit directly matches each impression against every credit row that references it and multiplies spend by the path count.
- Use a full outer join deliberately: a day with spend and no credit is a genuine CPA of infinity worth surfacing, and a day with credit and no spend means the touchpoint predates the spend window, which is a signal about window choice rather than a row to discard.
- Attributed conversions is SUM(credit_fraction), a fractional quantity, not COUNT(*). Counting rows credits one conversion once per touchpoint and inflates the total by the mean path length, which makes CPA look far better than it is.
- Flag recent days as immature. Spend is final within hours while credit keeps arriving for the model's click_window_days plus ingestion lag, so the most recent click_window_days plus roughly two days have CPA biased high and ROAS biased low. Read that window off the filtered rows rather than hard-coding a number: click_window_days is constant within a pinned (attribution_model, model_version), so MAX(click_window_days) over the pinned filter is a read of that constant. Verify the precondition first, because if MIN and MAX differ the column is not constant, the flag is being computed against a window that does not exist, and it has to come from the model configuration instead.
Worked solution 40 min
- Run SELECT attribution_model, model_version, MIN(click_window_days), MAX(click_window_days), COUNT(*) FROM attribution_credit GROUP BY 1, 2 first, so you know what is actually in the table before filtering to one pair, and so you have checked that the window really is constant inside the pair you are about to pin.
- Build credit AS (SELECT line_item_id, (touchpoint_ts AT TIME ZONE 'UTC')::date AS d, SUM(credit_fraction) AS conv, SUM(credited_value_usd) AS value, MAX(click_window_days) AS win FROM attribution_credit WHERE attribution_model = :model AND model_version = :version AND touchpoint_ts >= :range_start AND touchpoint_ts < :range_end GROUP BY 1, 2).
- Build spend AS (SELECT line_item_id, (served_ts AT TIME ZONE 'UTC')::date AS d, SUM(billed_price_micros_usd) / 1e6 AS spend FROM ad_impression WHERE is_billable AND served_ts >= :range_start AND served_ts < :range_end GROUP BY 1, 2).
- FULL OUTER JOIN on (line_item_id, d), coalescing both key columns in the select list so rows present on only one side keep their identity.
- Compute cpa = spend / NULLIF(conv, 0), roas = value / NULLIF(spend, 0), and is_mature = d < current_date - (win + 2), where win is the click_window_days read off the pinned model and version. On a spend-only day win is null, so is_mature is null rather than TRUE; decide whether to fall back to the model's configured window or to leave the flag unknown, and say which.
- Re-run the same range a week later and diff the CPA column to measure the actual maturation curve rather than assuming the lag.
Follow-up
- Last Tuesday's CPA improved 15% overnight with no change in delivery. What happened, and what would you have shipped to prevent the question?
- The same conversion has credit rows under last_click and under data_driven. Can you sum them? What do you give a client who wants one number?
- model_version changed on the 14th. How do you present a series that spans that boundary without implying a performance change?
How do you prioritize your work when you are hit with competing reques…
How do you prioritize your work when you are hit with competing requests from different business units?
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How do you define and calculate ROC-AUC and LogLoss, and why are these…
How do you define and calculate ROC-AUC and LogLoss, and why are these metrics critical in ad-tech applications?
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- How would you detect that the metric is being gamed rather than genuinely improving?
How would you optimize a Python data pipeline that is bottlenecked by …
How would you optimize a Python data pipeline that is bottlenecked by memory when processing large log files?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Fix the population and the time window before naming any metric.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Cut the variance of a market-level holdout with pre-period data
A 28-day holdout randomises 80 geo_dma markets 40/40; treated markets run the campaign and held-out markets get no delivery. The outcome is deduplicated conversion_event rows per market over the flight plus a 7-day window. Market sizes are heavily skewed and the unadjusted design detects only a 12% relative lift at 80% power. Twelve months of pre-period market conversions are available. Specify a variance-reduction plan, state the precision you expect, and state what would disqualify a covariate.
Approach
- Apply CUPED: define Y_adj = Y - theta*(X - mean(X)) where X is pre-period conversions for the same market and theta = Cov(Y,X)/Var(X), estimated on the pooled arms. Variance falls by a factor of (1 - rho^2), where rho is the correlation between pre-period and in-period outcomes.
- Estimate rho from history before the test runs, by regressing a past 28-day market window on its own preceding pre-period covariate. At rho = 0.9 the variance multiplier is 0.19, the standard error multiplier is 0.436, and the 12% MDE falls to roughly 5.2%.
- Disqualify any covariate measured at or after assignment. In-flight impressions, in-flight spend and in-flight clicks are all downstream of treatment; adjusting on them shifts the point estimate rather than only shrinking its variance.
- Stratify as well as adjust: form matched pairs on pre-period conversion volume, randomise within pair, and then analyse with pair fixed effects or a paired difference. Pairing that is not carried into the estimator buys nothing at all.
- Handle the skew at design time by analysing the ratio metric, conversions per thousand eligible bid requests, or by weighting on pre-period volume, and commit to the choice before seeing the arms.
- Pre-register theta's estimation rule and the covariate window, so neither can be tuned after the outcome becomes visible.
Worked solution 30 min
- On 12 months of history, build market-level 28-day outcome windows with matching pre-period covariates and compute rho; assume it returns 0.90.
- Compute the variance multiplier 1 - rho^2 = 0.19 and the standard error multiplier sqrt(0.19) = 0.436.
- Scale the unadjusted MDE: 12% * 0.436 = about 5.2% relative lift at 80% power, equivalent to 1/0.19 = 5.3 times the market count.
- Form 40 matched pairs on pre-period volume, randomise within pair, and write the estimator as a paired difference on CUPED-adjusted outcomes.
- Replay the whole pipeline as an A/A on two historical windows and confirm the adjusted estimate is centred on zero with the predicted standard error.
Follow-up
- CUPED moves the point estimate by 30% of its own standard error. What do you conclude?
- Would you use pre-period conversions or pre-period spend as the covariate, and why?
- How would the plan change if only 20 markets were available?
One advertiser's conversions doubled the week their SDK shipped
An advertiser's daily conversion_event rows roughly doubled starting 14 August, with no change in their order volume as they report it. That week they added a server_api feed alongside their existing browser_pixel. conversion_event has conversion_id, advertiser_id, conversion_ts, received_ts, conversion_type, order_id, device_id, user_id_hash, click_tracking_id, source, dedup_key and is_duplicate, where is_duplicate is written by our dedup job. Quantify the true conversion count, name the failure precisely, and state what you would tell the account team.
Approach
- Compare row counts against distinct dedup_key counts per advertiser per day. If rows double while distinct keys stay flat, the extra rows are duplicates that the dedup job never collapsed, and the real conversion level never moved.
- Establish why the job failed rather than that it failed: compute the null rate of dedup_key by source, and for non-null keys compare the value format between browser_pixel and server_api on the same order_id. Case, prefixing and whitespace differences are the usual cause and are invisible in a count of non-null keys.
- Use order_id as an independent second key. Distinct order_id per advertiser per day that is flat across 14 August confirms the real outcome count is unchanged and gives you the recovery rule.
- Size the blast radius downstream: duplicated conversions inflate attributed conversions, deflate CPA and inflate ROAS for every line item serving that advertiser, and the attribution_credit rows already written carry the inflation.
- Write the restatement rather than just the diagnosis — which advertiser, which dates, the direction and size of the correction, and whether any optimisation decision was already taken on the inflated numbers.
Follow-up
- The advertiser cannot change their server payload for a quarter. What dedup rule would you run in the meantime, and what does it get wrong?
- How would you detect this class of failure automatically for the other several thousand advertisers?
- If the two sources disagree on conversion_value_usd for the same order, which one wins and why?
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Practice prompt ↗Practice prompt ↗03Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Practice prompt ↗Practice prompt ↗04The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Practice prompt ↗Practice prompt ↗06Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Practice prompt ↗Practice prompt ↗07Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.
Tell me about a project where the data was extremely messy or incomple…
Tell me about a project where the data was extremely messy or incomplete. How did you unblock yourself to deliver results?
Approach
- State the situation in two sentences and spend the rest on your reasoning.
- Quantify the outcome, including what you would not claim credit for.
- Pick a story where you drove the decision, not one where you observed it.
Follow-up
- What did you decide not to do, and why?
- How did you know the outcome was caused by your change?
Defending a null incrementality result against a headline ROAS
Your geo holdout shows a retargeting line item's incremental conversions per $1,000 of spend are indistinguishable from zero. The same line item reports last-click ROAS of 8.3 and is the headline number in a renewal deck presented in six days. The account team believes your test is wrong; the advertiser has never questioned the 8.3. Deliverable: the internal case you make, the specific evidence you bring, the limits you concede, and the smallest reversible experiment you propose if your reading is rejected.
Approach
- The interviewer is probing whether you can hold an unpopular finding without turning it into a credibility fight, so open by granting that both numbers are computed correctly: last-click measures which touchpoint was nearest in time, the holdout measures what changed because of the spend. Framing attribution as 'wrong' converts a methodological point into a turf argument you will lose on relationship grounds.
- State the null as a bound rather than as zero. Report the minimum detectable effect the 20-market design could resolve and say 'we can rule out incremental ROAS above this value'. A null without an MDE is not a finding and is trivially dismissed as an underpowered test.
- Bring converging evidence from tables you already have, and say what each piece can and cannot show. Credit sensitivity: recompute the same conversion cohort under a second attribution_model, first-touch or even credit across in-window touchpoints, holding cohort and windows fixed, and report what share of the line item's credited volume survives; that bounds how much of the 8.3 is a property of the crediting rule rather than of the spend, and it is a sensitivity result, not a lift estimate. Harvesting signature: the share of retargeting-credited conversions whose user already had an earlier impression, click or site session inside click_window_days, together with the distribution of conversion_ts minus click_ts, where a heavy mass inside a few minutes is consistent with an ad served into a session that was already converting.
- Quantify both error directions before recommending anything: the spend at risk if you are right and ignored, and the renewal revenue at risk if you are wrong and act. This is what separates a defensible position from an ideological one.
- Propose the reversible version rather than the shutdown: a 20% budget reduction in half the matched markets for six weeks, powered against the MDE you just computed, with the analysis window extended past the flight by the click window plus ingestion lag.
Follow-up
- The account lead says telling the advertiser their favourite channel does nothing will cost us the renewal. How do you respond without either caving or escalating?
- What specific evidence would change your mind about this line item?
- How do you handle it if the budget-down test also comes back underpowered?
Scoping a one-line request that attribution is wrong
A sales lead forwards a one-line request: 'attribution is wrong for this advertiser, fix it.' You have read access to conversion_event, attribution_credit and ad_click for the account, and thirty minutes with the account manager. You may not contact the advertiser this week. Deliverable: a one-page scope naming the single question you will answer, the questions you are explicitly not answering, the data you need, and the decision the answer changes. Bring the three clarifying questions you would ask the account manager first.
Approach
- The interviewer is probing whether you convert a complaint into a comparison before you start work, so begin by writing the complaint as 'number A versus number B' and leave B blank until the account manager names it: attributed conversions against the advertiser's own order table, attributed CPA against last month, or our report against a second vendor's. Each has a different investigation.
- Run the two cheap arithmetic checks that resolve a large share of these before any modelling: confirm SUM(credit_fraction) per conversion_id equals 1.0 within one attribution_model and model_version, and measure the duplication rate by counting dedup_key values that appear with more than one distinct source in conversion_event.
- Measure identity coverage for the account: share of conversion_event rows carrying a non-null click_tracking_id and share carrying device_id. A low match rate means the channel is under-credited, which is the opposite complaint and needs the opposite fix.
- Decide the scope from what the answer will change. If the decision is budget reallocation, the honest scope is an incrementality read, not an attribution repair. If the decision is an invoice dispute, the scope is a reconciliation against the advertiser's order count.
- Write the page with an explicit out-of-scope list: model choice, window changes and anything requiring advertiser data you cannot get this week, each with the condition that would bring it back in scope.
Follow-up
- The account manager says the advertiser just wants the numbers to match. What do you tell them is actually achievable, and why is exact agreement not one of the options?
- How does your scope change if the discrepancy is 4% rather than 40%?
- 01
Tell me about a project where the data was extremely messy or incomplete. How did you unblock yourself to deliver results?
- 02
Your geo holdout shows a retargeting line item's incremental conversions per $1,000 of spend are indistinguishable from zero. The same line item reports last-click ROAS of 8.3 and is the headline number in a renewal deck presented in six days. The account team believes your test is wrong; the advertiser has never questioned the 8.3. Deliverable: the internal case you make, the specific evidence you bring, the limits you concede, and the smallest reversible experiment you propose if your reading is rejected.
- 03
A sales lead forwards a one-line request: 'attribution is wrong for this advertiser, fix it.' You have read access to conversion_event, attribution_credit and ad_click for the account, and thirty minutes with the account manager. You may not contact the advertiser this week. Deliverable: a one-page scope naming the single question you will answer, the questions you are explicitly not answering, the data you need, and the decision the answer changes. Bring the three clarifying questions you would ask the account manager first.
Is this an official OpenX interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at OpenX. Rounds and questions reflect what candidates have reported, not a process OpenX has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How much coding should I expect in the Data Scientist interview?
You should expect at least one dedicated coding round using a live sharing platform like codepad. The focus is on writing clean, optimal Python code to solve algorithmic and data manipulation problems, similar to medium-difficulty Leetcode challenges.
PracHub interview research ↗Does OpenX require prior experience in ad-tech?
While prior ad-tech experience is a significant advantage, it is not strictly required. However, you should take the time to learn the basics of programmatic advertising, real-time bidding, and common ad-tech metrics before your interview.
PracHub interview research ↗How does the team evaluate communication skills?
Communication is evaluated throughout the process, particularly in the behavioral round and the technical chat with the director. They look for your ability to explain complex machine learning choices simply and your capacity to collaborate across engineering and product teams.
PracHub interview research ↗What is the work environment and culture like for the data science team?
The culture is highly collaborative, data-driven, and fast-paced. Because the company processes such massive volumes of data, there is a strong emphasis on engineering excellence, continuous learning, and practical problem-solving.
PracHub interview research ↗Sources & methodology 3 sources ↗
Official role evidence, timestamped platform data and clearly labeled preparation advice.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
platform · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
platform · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
platform · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Truliant Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for Truliant data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.
tvScientific Data Scientist Interview Questions & Guide 2026
Timed practice for tvScientific data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
StackAdapt Data Scientist Interview Questions & Guide 2026
Experiment-design practice for StackAdapt data scientist candidates, covering randomisation units, power, minimum detectable effect and the traps that invalidate a result.
Taboola Data Scientist Interview Questions & Guide 2026
Take-home style practice for data scientists preparing for Taboola: a messy dataset, a fixed time budget, and a written analysis, reviewed for whether it states its assumptions and whether a reader could reproduce your numbers.