As a Data Scientist at PAPER, you will play a crucial role in harnessing data to enhance educational outcomes and drive strategic decisions. This position is pivotal as it not only influences the development of innovative educational products but also directly impacts the learning experiences of students and educators alike. You will be at the forefront of transforming raw data into actionable insights, guiding product development, and improving user engagement.
The work of a Data Scientist at PAPER involves collaborating closely with cross-functional teams, including engineering, product management, and operations. You will be tasked with analyzing complex datasets, developing predictive models, and providing strategic recommendations that shape the direction of various projects. The role is dynamic and rewarding, offering opportunities to tackle intriguing challenges that require both technical expertise and creative problem-solving skills.
In this position, you will engage with real-world data, driving initiatives that affect a wide range of stakeholders, from students and educators to product teams and executive leadership. Your contributions will be vital in ensuring that continues to deliver high-quality educational solutions tailored to the needs of its users.
Phone Screen
reportedWhoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.
What to demonstrate
- Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
- Whether your reason for wanting the role points at the work itself rather than the company's reputation
- Whether your language signals the level being screened for: what you decided yourself versus what you were handed
How to prepare
- Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
- Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
- Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands
Technical Interviews
reportedMuch of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.
What to demonstrate
- Whether the query you write matches the plan you just described
- What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
- Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly
How to prepare
- Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
- Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
- Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock
Leadership Interview
reportedAn extra round usually exists because something is still open after the standard loop: a skill the earlier interviews did not sample, a level decision, or two interviewers who disagreed. It is rarely a rerun of what you already did well. Ask the recruiter who you are meeting, what function they sit in, and how long the session runs. That is an ordinary scheduling question, and the answer changes what you should prepare. What separates a strong candidate here is treating the round as a fresh evaluation with its own bar, rather than assuming earlier performance carries you through or sinks you.
What to demonstrate
- Whether you can answer well on ground the earlier rounds did not cover, without leaning on what you already said to someone else
- Consistency of the facts in your stories: the same sample size, timeframe, team size and scope of your own role as in earlier conversations
- How you handle an unfamiliar format live, including whether you ask what kind of answer is wanted before producing one
How to prepare
- Ask the recruiter for the interviewer's function, the length, and whether to expect a coding surface, a discussion, or a presentation. Preparing for a 30 minute conversation with a partner team is not the same work as preparing for a 60 minute technical block.
- Write out what each earlier round actually covered, then list the two or three areas nobody probed. That gap is the most likely subject of the extra round.
- Re-read the numbers in the project stories you have already told, so a second telling does not quietly contradict the first.
PracHub editorial advice for the preparation topics above.
Pre/post gain studies that select on low pretest scores manufacture improvement.
Any measure with reliability below 1 produces regression to the mean, so a group chosen for scoring in the bottom quartile will score higher on retest with no intervention at all. The apparent gain scales with measurement error, which for a short quiz is large. The fix is a control group selected by the identical rule, or a design that models the pretest as a covariate rather than as a selection filter.
Roster sync creates and deactivates accounts in bulk, and it is not behaviour.
An automated roster import at term start can create thousands of accounts in one hour and deactivate thousands more at term end. Signup, activation, and churn series computed without filtering enrollment_source IN ('roster_sync','admin_bulk') will show spikes and cliffs driven entirely by an integration job. It also breaks cohort retention: a roster-created account that never activates is a provisioning artefact, not a churned learner.
Naming a model class before naming the deployment constraints
Set out the latency budget, the label delay, the retraining cadence, the interpretability requirement and the number of labelled examples, then pick the model that fits them. A boosted-tree answer to a problem where each decision must be explained to the affected user is a well-executed answer to the wrong question.
Averaging per-user rates to produce a population rate
Decide which quantity you want: the mean of per-user ratios and the ratio of summed numerator to summed denominator are different estimands, and heavy users dominate one but not the other. For a ratio metric, aggregate numerator and denominator separately and use the delta method for its variance.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
How would you optimize a machine learning model for better performance…
How would you optimize a machine learning model for better performance?
Approach
- Say how the offline result would be validated online before it is trusted.
- Set a baseline first, so any model has something honest to beat.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- What would you monitor after launch to know the model is still valid?
Simulate the gain a bottom-quartile selection manufactures
Observed quiz scores are a true score plus independent noise, both normal, with reliability r equal to true-score variance over observed-score variance. Simulate 200,000 learners, standardise the pretest to mean 0 and SD 1, select the bottom quartile on the pretest, and measure the mean pretest-to-posttest change with no intervention at all. Report the manufactured gain for r in {0.5, 0.7, 0.9, 1.0}, and give the closed form your simulation should reproduce. Use numpy only, no statistical packages.
Approach
- Parameterise by r directly: draw the true score with variance r and each noise term with variance 1 - r, so the observed score has unit variance and reliability exactly r by construction. Draw one true score and two independent noise terms per learner.
- Select on the pretest and only the pretest. Selecting on the true score, or on the average of the two measurements, removes the correlation between selection and the pretest's own noise, and the artefact disappears. That sensitivity is the lesson.
- Compute the mean of posttest minus pretest over the selected group. The true score cancels in the difference, so whatever remains is entirely the difference of two noise draws conditioned on the first being low.
- Check against the closed form. For jointly normal standardised scores, E[posttest | pretest] = r * pretest, so the expected gain is (1 - r) * |E[pretest | bottom quartile]|, and E[pretest | bottom quartile] = -phi(z_0.25) / 0.25 = -1.2711.
- Extend the script with a control group selected by the identical rule from an untreated population and show the difference of differences returns to zero. That is the design fix you would actually propose, not a caveat in a footnote.
Follow-up
- A published case study reports large gains specifically for learners who started in the bottom quartile. What do you ask for before believing any of it?
- The posttest is twice as long as the pretest, so its reliability is higher. Does the manufactured gain grow or shrink, and why?
- Give a design that measures a real effect on exactly this selected population without a randomised control.
Audit response rows against the versioned content dimension
Write a check function over fct_assessment_response (response_id, learner_id, content_item_id, content_version_no, attempt_no, submitted_at, scoring_mode, scored_at, score_points, max_points) and dim_content_item (content_item_id, version_no, status, published_at, retired_at). Return one row per check with check_name, failing_count, denominator, and up to five example response_ids. Cover at least four checks: version pairs missing from the dimension, responses submitted before published_at or after retired_at, attempt_no that is duplicated or not gapless from 1 within a learner and item, and rubric_human rows still unscored more than 5 days after submitted_at.
Approach
- Do the existence check as a left merge on the pair (content_item_id, content_version_no) to (content_item_id, version_no) with indicator=True. Joining on content_item_id alone both hides the defect you are looking for and fans rows out by the number of versions per item.
- Evaluate the publication window only on rows that matched, and treat NULL retired_at as open-ended. Report the two directions separately: an early submission points at a publishing race, a late one points at a retirement that never removed the item from a live form.
- For attempt_no, group by (learner_id, content_item_id) and compare the sorted attempts to range(1, n+1). A duplicate and a gap are different defects with different causes, so emit them as two rows.
- For scoring lag, restrict to scoring_mode = 'rubric_human' with scored_at null and submitted_at older than 5 days. Anything more recent is expected lag, not a defect, and counting it makes the check permanently red.
- Give every check its own denominator. A shared total row count makes the version check look negligible and the attempt check look enormous when neither is true, because their natural units are rows and (learner, item) groups respectively.
Worked solution 20 min
- merged = responses.merge(dim, left_on=['content_item_id','content_version_no'], right_on=['content_item_id','version_no'], how='left', indicator=True); missing = merged._merge == 'left_only'.
- On matched rows: early = submitted_at < published_at; late = retired_at.notna() & (submitted_at > retired_at).
- attempts = responses.groupby(['learner_id','content_item_id']).attempt_no.agg(['count','nunique','max']); duplicated = nunique < count; gapped = (nunique == count) & (max != count).
- lagged = (scoring_mode == 'rubric_human') & scored_at.isna() & (submitted_at < now - 5 days).
- Assemble a frame of check_name, failing_count, denominator, and head(5) of the failing response_ids per check.
Follow-up
- Which of these four would you page someone about at 2am, and which is a ticket for the week?
- How do you keep this check from timing out once the fact table is in the billions of rows, without weakening what it detects?
Write code that can clean and preprocess a dataset for analysis.
Write code that can clean and preprocess a dataset for analysis.
Approach
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Say which table is the grain you start from, and join outward from it.
Follow-up
- How would you verify this result without re-running the same query?
- How does the query change if the join becomes one-to-many?
Can you explain time complexity and provide examples of different comp…
Can you explain time complexity and provide examples of different complexities?
Approach
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- Say which table is the grain you start from, and join outward from it.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
Follow-up
- What breaks if events arrive late or out of order?
- How does the query change if the join becomes one-to-many?
Rebuild sessions and active seconds from heartbeat events
Given raw_heartbeat(learner_id, content_item_id, content_version_no, heartbeat_at TIMESTAMP UTC, session_hint VARCHAR NULL), reconstruct the grain of fct_lesson_activity: one row per learner per content item per session. Close a session after 30 minutes with no heartbeat. Compute active_seconds as the sum of inter-heartbeat gaps inside a session with each gap capped at 120 seconds, and wall_seconds as last minus first heartbeat. Ignore session_hint, which the client sets unreliably. Then reconcile your active_seconds against the stored column in fct_lesson_activity and explain any systematic difference.
Approach
- Compute prev_at = LAG(heartbeat_at) OVER (PARTITION BY learner_id, content_item_id, content_version_no ORDER BY heartbeat_at) and gap_seconds from the difference. Partition on the version too, since the same item at a new version is different content.
- Flag is_new_session = (prev_at IS NULL OR gap_seconds > 1800), then session_seq = SUM(is_new_session::int) OVER (same partition ORDER BY heartbeat_at ROWS UNBOUNDED PRECEDING). The running sum over an ordered flag is the island assignment and it stays correct at any data volume, unlike a self-join on nearby timestamps.
- Aggregate per (partition, session_seq): active_seconds = SUM(CASE WHEN is_new_session THEN 0 ELSE LEAST(gap_seconds, 120) END), wall_seconds = MAX(heartbeat_at) - MIN(heartbeat_at), plus heartbeat_count.
- State the two consequences of the definition before anyone asks: a single-heartbeat session has active_seconds = 0 and wall_seconds = 0, and the per-gap cap means active_seconds can never exceed 120 * (heartbeat_count - 1), so a client with a slow heartbeat interval understates real time.
- Reconcile by joining on (learner_id, content_item_id, started_at) and examining the distribution of the difference, not a row count. A uniform shift points at the cap or the close rule; a heavy one-sided tail points at heartbeats the stored column excludes.
Worked solution 30 min
- Write the LAG and gap CTE and eyeball ten rows for one busy learner before adding anything else.
- Add the is_new_session flag and the running-sum session_seq, then verify session_seq increments exactly where the gap exceeds 1800 seconds.
- Aggregate to the session grain with the CASE-guarded capped sum.
- Join to fct_lesson_activity on learner, item and started_at, and plot or bucket the signed difference in active_seconds.
Follow-up
- The stored column excludes backgrounded tabs but raw_heartbeat has no visibility flag. How would you detect backgrounded stretches from timing alone, and how confident should you be?
- Two heartbeats can share a timestamp. What does that do to LAG, and does it change the answer?
- A learner studies for two hours with a 40-minute dinner break in the middle. Your rule gives two sessions. Which downstream metrics care about that choice and which do not?
How would you approach building a recommendation system for an educati…
How would you approach building a recommendation system for an educational platform?
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How do you ensure the integrity and reliability of your data analyses?
How do you ensure the integrity and reliability of your data analyses?
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
Given a dataset of student performance, what steps would you take to a…
Given a dataset of student performance, what steps would you take to analyze it?
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Restate the decision this analysis has to support, and who acts on the answer.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
If you were tasked with improving user engagement metrics, what data w…
If you were tasked with improving user engagement metrics, what data would you consider?
Approach
- Fix the population and the time window before naming any metric.
- State what result would change your recommendation, so the answer is falsifiable.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Describe how you would design an experiment to test a new feature in a…
Describe how you would design an experiment to test a new feature in an educational app.
Approach
- Name the randomisation unit first; it decides the variance and what the test can detect.
- Name the guardrails that would stop a launch even on a positive primary result.
- State the primary metric and the minimum effect worth shipping, then size the test.
Follow-up
- How would you handle interference between treated and control units?
- What would you do if you could not randomise at all?
What statistical methods do you prefer for analyzing large datasets?
What statistical methods do you prefer for analyzing large datasets?
Approach
- State your assumptions explicitly before working the problem.
- Work from the decision backwards to the evidence you would need.
- Say what you would check first and why it is the highest-information step.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Weekly engagement metric robust to term calendars
Weekly active learners fell 42% week over week. The prior week was term-end and the current week contained a holiday. Using fct_lesson_activity (activity_date, learner_id, active_seconds), fct_enrollment (enrollment_source, status) and a term dimension mapping each date to a term, a term-week index and an instructional-day flag, write a weekly engagement metric definition a stakeholder can read week to week. Specify the numerator, the denominator, the alignment rule, and which enrollment sources you exclude and why.
Approach
- Refuse the comparison before defining anything: a term-end week and a holiday week are not comparable units, so the -42% is a calendar fact, not a product signal.
- Define the numerator with a floor: distinct learner_id in fct_lesson_activity with at least 60 active_seconds inside the term-week, so a stray page open does not count as engagement.
- Define the denominator deliberately: learners with an active enrollment in that term-week, excluding accounts created by roster_sync or admin_bulk that have never had first_activity_at set. Those are provisioning artefacts from an integration job, not dormant learners, and leaving them in makes every rate move when a district runs a sync.
- Handle short weeks explicitly: report the rate per instructional day, or restrict to full instructional weeks and label the short ones on the chart, rather than averaging a three-day week into the series.
- Compare term-week W of this term to term-week W of the prior term through the term dimension, never to the adjacent calendar week.
- Publish the definition string beside the series and state the age_gated share of the population, since behavioural logging is restricted for those learners and the series is missing them by construction.
Worked solution 20 min
- Join activity_date to the term dimension to get (term_id, term_week, is_instructional_day) for every row.
- Write the numerator with the 60-second floor and the denominator with the active-enrollment rule and the roster_sync and admin_bulk exclusion.
- Add the per-instructional-day normalisation and the same-term-week prior-term comparison.
- Recompute the reported drop under the aligned definition and state what remains after the calendar is removed.
Follow-up
- Two institutions run terms of different lengths. How do you align term-week 6 between them?
- A district runs its roster sync two weeks late. What does your series do, and what would you want it to do?
- What would you show a stakeholder who insists on a week-over-week number anyway?
Net revenue retention slid nine points in one quarter
Net revenue retention on institutional licenses fell from 104 percent to 95 percent quarter over quarter. Source: fct_subscription_period (subscription_period_id, account_id, org_id, plan_tier, billing_interval, seats_purchased, seats_provisioned, mrr_usd, period_start, period_end, is_renewal, status, canceled_at, churn_reason). Decompose the nine points into expansion, contraction and non-renewal, and establish whether renewal behaviour changed at all. Deliverable: the decomposition, the cohort definition you used, and a call on whether this is a trend or account news.
Approach
- Pin the cohort in writing before computing anything: orgs holding an active period twelve months before each renewal date, new logos excluded from both numerator and denominator, and non-renewals entering the numerator as zero rather than being dropped. Dropping non-renewals is the most common way this metric is overstated, and correcting it can move the headline by more than the effect under investigation.
- Rebuild the ratio from components on that cohort: starting annualised revenue, expansion, contraction, non-renewal. Check the components sum back to the reported ratio in both quarters; a residual means the two quarters were not computed on the same cohort rule and the comparison is void before any interpretation.
- Test cohort comparability on billing_interval. Multi-year contracts do not present a renewal in every window, so a quarter whose renewal cohort holds a different multi_year share is not comparable to its predecessor. Recompute both quarters holding the interval mix fixed and report that alongside the raw number.
- Rank per-org contributions to the nine points and state how much the top three carry. Institutional revenue is concentrated, so a decomposition that leaves the concentration unstated invites a trend reading of what is account news.
- Audit the inputs: confirm mrr_usd normalisation is consistent (annual divided by twelve) and look for overlapping period_start and period_end within a single account_id, which double counts revenue after a mid-term upgrade.
- Close the loop to something actionable by joining the lost orgs back to seat activation and pace adherence in the preceding term, and say whether the signal was visible early enough for anyone to have acted on it.
Follow-up
- Build the renewal-risk view from this: which signals are actionable while there is still time to act, and at what days-to-renewal would you surface an org?
- An org contracts from 900 seats to 600 but moves to a higher tier and raises MRR. Expansion or contraction, and does your decomposition handle it without double counting?
- How would you present NRR so that a quarter with an unusual renewal cohort is not read as a trend by an executive audience?
Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Fix the scope and set a baseline
- Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
- Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
- Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.
Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02One query pattern, written three times
- Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
- On the third attempt, write the grain of every CTE as a comment before writing its body.
- Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.
Deliverable: Three independent versions of the same query plus a note on what changed between them.
Practice prompt ↗Practice prompt ↗Practice prompt ↗03Only the statistics you will be asked to defend
- Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
- Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
- Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.
Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.
Practice prompt ↗Practice prompt ↗Practice prompt ↗04One case, and the assumptions holding it up
- Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
- Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
- Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.
Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Your own work, timed
- Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
- Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
- Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.
Deliverable: Two timed narratives with one defensible number in the opening line.
Practice prompt ↗Practice prompt ↗06The one full rehearsal, in a longer weekend block
- Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
- Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
- Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.
Deliverable: Mock notes naming three failure moments with a specific fix written under each.
Practice prompt ↗Practice prompt ↗07Taper
- Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
- Re-read only your own notes from this week, and open no new material.
- Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.
Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Work that nobody used is a common and unflattering pattern in data careers, and interviewers probe for it. Have a story about an analysis that changed a decision, and be specific about how you got it in front of the person who could act. Also have one about work that went nowhere, with your reading of why.
How do you handle missing data in your analyses?
How do you handle missing data in your analyses?
Approach
- State the situation in two sentences and spend the rest on your reasoning.
- Pick a story where you drove the decision, not one where you observed it.
- Quantify the outcome, including what you would not claim credit for.
Follow-up
- What would you do differently if you ran that project again?
- What did you decide not to do, and why?
Disagree with a product manager using evidence, not volume
A product manager wants to ship an adaptive practice selector to all grade bands on the strength of a four-point rise in first-attempt accuracy during a six-week pilot. You believe the rise is an artefact of the selector's target success rate. You have fct_assessment_response, dim_content_item with irt_a, irt_b and calibration_n, and the pilot arm assignment. Deliverable: the analysis that tests your objection, and how you present it so the PM can change position without it reading as a defeat. Probed: whether you make disagreement falsifiable rather than rhetorical.
Approach
- Make the objection falsifiable before raising it. The claim implies two testable predictions: mean calibrated difficulty of served items rose with learner ability, and accuracy is flat within ability strata.
- Compute weekly mean irt_b of served items per arm, restricted to items with calibration_n above your floor, and plot it against the accuracy series. If served difficulty tracked ability, the accuracy line carries no learning signal and you can show that rather than assert it.
- Build the metric that survives adaptivity: a small fixed-form set with (content_item_id, version_no) held constant, served to both arms, reported as the pilot's accuracy readout.
- Bring the replacement to the meeting, not only the refutation. A PM who has been told the number is meaningless still has a launch decision and no instrument.
- Separate the two questions out loud: whether the selector helps learners is open and testable; whether first-attempt accuracy measures it is settled, and it does not.
Follow-up
- The fixed-form set costs each learner six minutes a fortnight. How do you justify that to the same PM?
- Mean served irt_b is flat but accuracy still rose four points. What do you look at next?
State the impact of your last project without inflating it
Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.
Approach
- Name the metric by its definition rather than its label. 'Verified mastery rate' means nothing without its numerator, denominator, window, and the reporting lag the delayed retention check forces.
- Separate the three claims usually merged into one: the metric moved, your work moved it, and the movement was worth what it cost.
- State the counterfactual explicitly and say what produced it: a holdout, matched sections, or a pre-period trend. If it was a before-and-after with no control, say that in those words.
- List the co-occurring changes with dates: a term boundary, a roster sync at scale, a pricing change, another team shipping into the same surface. In this domain the academic calendar alone routinely dwarfs a product effect.
- Convert the shortfall into a specific design artefact: the holdout you would have reserved, the instrumentation you would have shipped before the change, the analysis plan you would have written first.
Follow-up
- What fraction of the observed movement would you defend under cross-examination, and on what evidence?
- Your project shipped in term-week one. How does that change what you can claim?
- If the effect is genuinely not separable, was the project worth doing?
- 01
How do you handle missing data in your analyses?
- 02
A product manager wants to ship an adaptive practice selector to all grade bands on the strength of a four-point rise in first-attempt accuracy during a six-week pilot. You believe the rise is an artefact of the selector's target success rate. You have fct_assessment_response, dim_content_item with irt_a, irt_b and calibration_n, and the pilot arm assignment. Deliverable: the analysis that tests your objection, and how you present it so the PM can change position without it reading as a defeat. Probed: whether you make disagreement falsifiable rather than rhetorical.
- 03
Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.
Is this an official PAPER interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at PAPER. Rounds and questions reflect what candidates have reported, not a process PAPER has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult are the interviews, and how much preparation time should I expect?
The interviews are moderately challenging, requiring a solid understanding of data science concepts and problem-solving skills. Candidates typically spend several weeks preparing, focusing on both technical and behavioral aspects.
PracHub interview research ↗What differentiates successful candidates?
Successful candidates often demonstrate a strong technical foundation, effective communication skills, and a collaborative mindset. Showing enthusiasm for PAPER's mission and values also makes a significant difference.
PracHub interview research ↗What is the culture and working style at PAPER?
PAPER fosters a collaborative and innovative culture, encouraging team members to share ideas and work together toward common goals. Flexibility and adaptability are highly valued, as the company is dedicated to improving educational outcomes.
PracHub interview research ↗What is the typical timeline from initial screen to offer?
The interview process usually takes about a month, depending on scheduling and availability. Candidates should expect several stages, including phone screens and technical interviews.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Yale University Data Scientist Interview Questions & Guide 2026
Timed practice for Yale University data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
Universitätsklinikum Tübingen Data Scientist Interview Questions & Guide 2026
A prep plan for Universitätsklinikum Tübingen data scientist candidates built on spaced review: write a full answer before opening any solution, keep a running list of the assumptions you failed to state, and repeat every missed exercise th
University of Michigan Data Scientist Interview Questions & Guide 2026
Metric-diagnosis practice for University of Michigan data scientist candidates: a number moves week over week and you work out why, splitting by segment, ruling out a logging or release change, and separating a mix shift from a real change
University of Florida Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for University of Florida data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.