As a Data Scientist at Khan Academy, you play a foundational role in fulfilling the organization's mission to provide a free, world-class education for anyone, anywhere. Operating at the intersection of product development, learning science, and data engineering, you empower cross-functional teams to make evidence-based decisions that directly impact over 181 million registered learners globally. Your day-to-day work involves transforming massive streams of educational and behavioral data into actionable insights, driving product strategies that optimize student engagement and improve learning outcomes.
The scope of this role extends across multiple high-impact product areas, including the core learner experience, content recommendation engines, and teacher-parent tools. You will partner closely with product managers, engineers, and learning scientists to design theories of action, instrument robust event tracking, and quantify the real-world impact of new features. Because Khan Academy operates as a mission-driven nonprofit, your work carries a unique weight: every metric drop you diagnose and every successful experiment you analyze directly translates to better educational access and equity for students in historically under-resourced communities.
Succeeding in this role requires a rare blend of rigorous technical capability and deep empathy for learners and educators. You must be comfortable navigating messy, large-scale behavioral datasets while maintaining a clear view of the pedagogical goals behind the numbers. If you are a curious, highly collaborative practitioner who wants to apply advanced statistical modeling and experimentation frameworks to redefine the future of education, this role offers an unmatched platform for professional impact.
Recruiter Screen
reportedData Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.
What to demonstrate
- Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
- Whether you name what you have not done instead of stretching to cover every line of the posting
- Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage
How to prepare
- Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
- Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
- Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live
Technical Evaluation
reportedThis round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.
What to demonstrate
- Whether your row counts survive each join, and whether you notice on your own when they do not
- Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
- Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
- Reaching a defensible answer inside the window instead of a refined one after it
How to prepare
- Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
- Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
- Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither
SQL and Product Assessment
reportedA handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.
What to demonstrate
- Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
- Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
- Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.
How to prepare
- Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
- Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
- If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows
Cross-Functional Interviews
reportedAn added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.
What to demonstrate
- Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
- Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
- How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact
How to prepare
- Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
- For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
- Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.
Final Rounds
reportedWhere a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.
What to demonstrate
- Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
- Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
- Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
- Whether you ask what decision is actually on the table before explaining anything
How to prepare
- Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
- For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
- Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else
PracHub editorial advice for the preparation topics above.
Consent and age rules silently truncate the data, not just the joins.
Where age_gated is TRUE, behavioural logging and cross-system joins are restricted, so those learners are missing from the very tables used to compute engagement. An analysis that simply inner-joins will produce a population skewed to older learners and self-serve accounts while appearing complete. Check the age_gated share of every population you report on, and state it.
Pre/post gain studies that select on low pretest scores manufacture improvement.
Any measure with reliability below 1 produces regression to the mean, so a group chosen for scoring in the bottom quartile will score higher on retest with no intervention at all. The apparent gain scales with measurement error, which for a short quiz is large. The fix is a control group selected by the identical rule, or a design that models the pretest as a covariate rather than as a selection filter.
Extrapolating a first-week lift inflated by novelty effects
Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.
Accepting a metric definition without asking about the denominator
Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
How do you check for confounding variables when analyzing observationa…
How do you check for confounding variables when analyzing observational student learning data?
Approach
- Write down the assumption the method needs before you use the method.
- Sanity-check the answer against a simple bound or a simulated case.
- Translate the result into the decision it informs, in one plain sentence.
Follow-up
- How would you explain this result to someone who does not know statistics?
- Which assumption here is most likely to be violated in practice?
Describe how you would apply regression discontinuity design to evalua…
Describe how you would apply regression discontinuity design to evaluate the impact of a grade-advancement threshold on student retention.
Approach
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Check what information would not exist at prediction time, and exclude it.
Follow-up
- What would you monitor after launch to know the model is still valid?
- Where could label leakage enter this setup?
Sessionise a raw heartbeat stream into activity rows
You get a DataFrame heartbeats with columns learner_id, content_item_id, content_version_no, and ts (UTC timestamps, unsorted). Produce one row per learner per session per content item with started_at, ended_at, wall_seconds, and active_seconds. A session closes after 30 minutes with no heartbeat from that learner. active_seconds sums consecutive gaps inside a session, capping each gap at 120 seconds. Attribute each gap to the content item of the earlier heartbeat. State your rule for a gap of exactly 1800 seconds and for a session with a single heartbeat.
Approach
- Sort by learner_id then ts, and compute the gap with a per-learner diff so the first heartbeat of each learner has no gap. A global diff lets one learner's last heartbeat set the next learner's first gap.
- Flag a session break where gap > 1800 seconds, then cumsum that boolean within the learner to get a session index. Say whether exactly 1800 opens a new session; either choice is defensible but it must be stated because it moves the session count.
- Each gap covers the interval before the current heartbeat, so attribute min(gap, 120) to the content_item_id of the previous row via shift, not the current row.
- Group by (learner_id, session_index, content_item_id): sum the attributed capped gaps for active_seconds, take min and max ts for started_at and ended_at, and derive wall_seconds from those two.
- Handle the degenerate group explicitly. A single heartbeat gives active_seconds 0 and wall_seconds 0. Dropping those rows quietly removes real opens and biases any completion or abandonment rate computed later.
- Validate that within every session the summed active_seconds never exceeds the session's wall_seconds, which catches attribution and cap errors in one assertion.
Worked solution 35 min
- Sort by (learner_id, ts) and compute gap = ts.diff().dt.total_seconds(), then null the gap on the first row of each learner using a learner-change mask.
- new_session = gap.isna() | (gap > 1800); session_index = new_session.groupby(learner_id).cumsum().
- contribution = gap.clip(upper=120).fillna(0), attributed to the previous row's content_item_id using shift within the learner and session.
- Aggregate by (learner_id, session_index, content_item_id) with sum for active_seconds and min and max of ts for the timestamps.
- Assert per session that sum(active_seconds) <= (max(ts) - min(ts)) in seconds, and print the count of single-heartbeat groups.
Follow-up
- The client emits a heartbeat every 60 seconds on desktop and every 15 seconds on a managed device fleet. What does the 120-second cap do to the comparison between those two populations?
- Your reconstructed active_seconds disagrees with the stored column by 4 percent overall. How do you localise the disagreement rather than argue about the total?
- A learner has two tabs open on different items at once. What does your output claim, and what should it claim?
Write a SQL query using window functions to calculate the 7-day rollin…
Write a SQL query using window functions to calculate the 7-day rolling average of active learners per grade level.
Approach
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- State the window function and its partition and ordering out loud before writing it.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
Follow-up
- How would you verify this result without re-running the same query?
- How does the query change if the join becomes one-to-many?
How would you use a self-join or window function to identify consecuti…
How would you use a self-join or window function to identify consecutive days of platform usage for students?
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Say which table is the grain you start from, and join outward from it.
Follow-up
- How does the query change if the join becomes one-to-many?
- What breaks if events arrive late or out of order?
Consecutive instructional-week streaks and term-over-term return
Using fct_lesson_activity(learner_id, enrollment_id, activity_date, active_seconds) and a calendar dim_term_week(term_id, term_week_no, week_start_date, week_end_date, is_instructional), where holiday weeks carry is_instructional = FALSE, compute for each learner and term the longest run of consecutive instructional weeks containing at least one activity day. A non-instructional week must not break a run, and activity inside one neither extends nor starts a run. Then compute term-over-term return rate into term T+1 with a floor of five activity days in term T. Return learner_id, term_id, activity_days counted over the whole term including non-instructional weeks, longest_streak_weeks and the return flag.
Approach
- Collapse activity to (learner_id, term_id, term_week_no) with COUNT(DISTINCT activity_date). fct_lesson_activity is one row per item per session, so a learner with twelve rows on one afternoon is one activity day and counting rows inflates everything downstream.
- Re-index before differencing. Apply DENSE_RANK() OVER (PARTITION BY term_id ORDER BY term_week_no) across instructional weeks only, producing a gap-free sequence where a holiday week has simply been removed. Differencing raw term_week_no instead splits a run at every holiday, which is exactly the artefact the task forbids.
- Accept what dropping those weeks costs. Activity in a non-instructional week is removed from the streak calculation entirely, so a learner whose only activity lands in a holiday week has activity_days >= 1 and longest_streak_weeks = 0. That is what a streak over instructional weeks means, but it is why the two columns can disagree and why the checks must expect a zero streak alongside non-zero activity rather than assert a floor of 1.
- Apply the island trick on the re-indexed sequence: instructional_index - ROW_NUMBER() OVER (PARTITION BY learner_id, term_id ORDER BY instructional_index) is constant within a run. Group on that constant, count rows per group, take the max per learner and term.
- Build activity_days from the unfiltered term aggregate, then LEFT JOIN the streak result onto it and COALESCE longest_streak_weeks to 0. An inner join here deletes the holiday-only learners from the output and from the return-rate denominator, which is the same bug as the streak-floor assertion wearing different clothes.
- Set the activity floor on COUNT(DISTINCT activity_date) >= 5 across term T, applied to the return-rate denominator only. The floor exists to remove provisioned-but-unused accounts, so leaking it into the numerator silently redefines the metric as active in both terms. Note that a learner can clear the floor on holiday-week activity alone and enter the denominator with a zero streak.
- Define the return flag as EXISTS any activity row in term T+1 for that learner, pairing terms by an explicit term ordering rather than date arithmetic, since term lengths differ. Learners whose org has no term T+1 loaded must be excluded and counted, not treated as non-returners.
Worked solution 40 min
- Build the learner-week activity CTE with COUNT(DISTINCT activity_date) and confirm one row per (learner_id, term_id, term_week_no).
- Build the instructional re-index from dim_term_week and join activity onto it, dropping non-instructional weeks entirely.
- Apply the index-minus-row-number island grouping and take MAX(run_length) per learner and term.
- Compute activity_days per learner and term over the whole term, LEFT JOIN the streak onto it with COALESCE to 0, and count the learners who come out with activity_days > 0 and longest_streak_weeks = 0.
- Apply the five-day floor to the denominator and attach the term T+1 existence flag.
- Build a two-row fixture by hand and verify the holiday behaviour before trusting the full run.
Follow-up
- This streak rewards a learner doing one minute a week over one doing four hours in a single week. Which of those is the product claim, and what would you pair the streak with to catch the difference?
- Roster sync deactivates accounts in bulk at term end. How does that interact with the five-day floor and with the return denominator?
- Show how return rate moves as the floor goes from one day to five, and argue which floor is the honest one to publish.
If we wanted to expand our reach in under-resourced school districts, …
If we wanted to expand our reach in under-resourced school districts, what engagement metrics should our product team prioritize?
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
- Fix the population and the time window before naming any metric.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Define a set of north-star and proxy metrics for a newly introduced ga…
Define a set of north-star and proxy metrics for a newly introduced gamified learning streak feature.
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- How would you detect that the metric is being gamed rather than genuinely improving?
A teacher dashboard feature has been launched to increase classroom en…
A teacher dashboard feature has been launched to increase classroom engagement. What primary and secondary metrics would you track to determine its efficacy?
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
How would you evaluate whether a self-paced video module is actually d…
How would you evaluate whether a self-paced video module is actually driving deeper conceptual understanding versus passive viewing?
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Fix the population and the time window before naming any metric.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
Explain the concept of statistical significance and how you would expl…
Explain the concept of statistical significance and how you would explain Type I and Type II errors to a non-technical product manager.
Approach
- Decide the analysis before seeing data, including how long it runs and when you look.
- Name the randomisation unit first; it decides the variance and what the test can detect.
- State the primary metric and the minimum effect worth shipping, then size the test.
Follow-up
- What would you do if you could not randomise at all?
- How would you handle interference between treated and control units?
This category focuses on your capability to design, execute, and inter…
This category focuses on your capability to design, execute, and interpret controlled experiments while avoiding common statistical traps.
Approach
- Decide the analysis before seeing data, including how long it runs and when you look.
- Name the randomisation unit first; it decides the variance and what the test can detect.
- Name the guardrails that would stop a launch even on a positive primary result.
Follow-up
- What would you do if you could not randomise at all?
- What would you conclude if the result is positive but the test is underpowered?
Decide what an administrator usage report counts
You must publish a five-line usage report that institutional administrators read before renewal. Once it is published, instructors will assign work to move whatever it counts. Available: fct_lesson_activity (active_seconds, is_assigned, completion_status), fct_assessment_response (max_points, scoring_mode, is_correct, response_seconds, hint_count), fct_enrollment (units_completed, units_total) and dim_learner (first_activity_at, age_gated). Choose the five lines and their source columns, name one metric you refuse to publish and the behaviour it would cause, and state the pairing that keeps each line honest.
Approach
- Start from behaviour rather than from data availability. Publishing a number to the buyer converts it into an assignment, so the test for each line is whether its gamed version is still something you want happening.
- Choose lines that survive that test: seat activation with the never-activated count shown rather than hidden; learners with at least one scored submission in the last four weeks; pace adherence at the current term-week; verified mastery events with the reporting lag printed; and objectives with no coverage yet, which is a gap list, is directly actionable, and cannot be inflated.
- Name the source columns per line so the report is reproducible: first_activity_at against seats_provisioned, fct_assessment_response.submitted_at with scoring_mode, units_completed over units_total against the term dimension, the mastery ledger with its delayed check, and the objective coverage join.
- Refuse active minutes per learner. Publishing it causes seat-time assignments, which is the clearest case of teaching to the number in this domain and the weakest claim the product can defend as learning. Keep it internally as a guardrail where it is useful and harmless.
- Pair every countable line with an integrity partner: pace adherence with assessment integrity (response_seconds above the item floor, hint_count below cap), submissions with mean max_points so splitting one task into five cannot move the line, activation with the raw never-activated count.
- State the population on the report itself, including the age_gated share and any excluded enrollment sources, because those learners are missing from the behavioural tables and the report will otherwise look complete while describing a partial population.
Worked solution 30 min
- For each candidate line, write the one sentence describing how an instructor would move it with no learning, and drop the lines whose gamed version is undesirable.
- Write the five surviving lines with their source columns and windows.
- Attach an integrity or substance partner to each countable line, and write the single sentence justifying the refusal of active minutes per learner.
- Add the population footer: age_gated share, excluded enrollment sources, and the reporting lag on verified mastery.
Follow-up
- An administrator says your report shows fewer active learners than their own dashboard. What are the three likeliest causes?
- A district asks for a per-teacher leaderboard built from these lines. What do you do?
- If you could publish only four lines, which one goes, and what breaks?
Pace adherence at term-week six rose eleven points
Pace adherence at term-week 6 rose from 48 percent to 59 percent between two consecutive terms, and the dashboard query changed owners last month. Tables: fct_enrollment (enrollment_id, learner_id, course_id, units_completed, units_total, status, scheduled_start_date, due_date, enrolled_at) and dim_term (term_id, term_start, term_length_weeks). Nothing shipped to pacing in that period. Decide whether learners are pacing better, and reconcile the old and new numbers to the point.
Approach
- Diff the definitions before touching the data. Recover the prior query and compare it line by line against the current one on four axes: the denominator population, the status filter, how term-week 6 is derived, and what happens to enrollments with NULL scheduled_start_date.
- Recompute both definitions on the same term and the same enrollment set. If the difference does not account for the full eleven points there is a second cause, and knowing that before you start hunting saves the hunt.
- Test the survivorship mechanism directly: count enrollments with status in ('dropped','expired') at the end of term-week 6 and compute their units_completed over units_total. If that ratio sits well below the active median, removing them from the denominator lifts the rate mechanically and the lift is arithmetic, not behaviour.
- Verify the numerator independently, because units_completed is maintained incrementally rather than derived on read. Recount it from lesson activity for a sample of a couple of hundred enrollments and compare against the stored column to rule out a backfill as a second mover.
- Confirm term-week 6 maps to the intended dates in both terms through dim_term rather than calendar week, since an edited term_start silently shifts the evaluation window and can move the number on its own.
Follow-up
- The old definition penalises a course for enrollments that dropped in week two. Is that the behaviour you actually want from a pacing metric, and who is the metric for?
- How do you treat enrollments with NULL scheduled_start_date, and what does each choice do to the reported number?
Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Diagnostic, scored before you study anything
- Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
- Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
- Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.
Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02Largest gap: find the boundary rather than the subject
- Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
- Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
- Re-solve one of them from memory four hours later, on paper, with nothing open.
Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.
Practice prompt ↗Practice prompt ↗Practice prompt ↗03Largest gap: drill the blocking sub-skill
- Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
- Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
- Have someone else read your one-sentence rule and find the precondition you left out.
Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.
Practice prompt ↗Practice prompt ↗Practice prompt ↗04Second gap, plus maintenance on your strongest area
- Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
- Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
- Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.
Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.
Practice prompt ↗Practice prompt ↗Worked solution ↗05The gap that is not a skill
- Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
- Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
- Deliver one answer again with a hard 90-second limit to force structure before detail.
Deliverable: Two recordings with a counted improvement in time-to-first-question.
Practice prompt ↗Practice prompt ↗06Retest under day-one conditions
- Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
- Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
- Write which single block you would still lose the offer on.
Deliverable: A second scored rubric placed next to the first, with one named remaining risk.
Practice prompt ↗Practice prompt ↗07Full loop under interview conditions
- Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
- Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
- Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.
Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.
How do you handle sample ratio mismatch (SRM) when you discover it in …
How do you handle sample ratio mismatch (SRM) when you discover it in an ongoing educational experiment?
Approach
- Close with what you would do differently, concretely.
- Pick a story where you drove the decision, not one where you observed it.
- Name the disagreement or constraint, and how you resolved it with evidence.
Follow-up
- What did you decide not to do, and why?
- How did you know the outcome was caused by your change?
Explain a risk score to a non-technical sales executive
Your renewal-risk score joins fct_subscription_period to seat activation from dim_learner.first_activity_at and to pace adherence from fct_enrollment. For one institution with 900 seats_provisioned and 31% activation at term-week 6, it outputs a 0.62 probability of non-renewal. A sales VP asks whether you are losing the account or not. You have two minutes and no slides. Deliverable: the spoken answer, plus the one line you would put in the weekly account review so the number is not later quoted as a certainty. Probed: whether you can carry uncertainty and still give a decision.
Approach
- Answer the question that was asked before qualifying it. 'Treat this as at-risk and act this week' is the answer; the probability is the support for it.
- Translate the score into a frequency on a reference class the VP already trusts, such as how many of the last fifty accounts scored in the 0.55-0.70 band did not renew, instead of explaining calibration in the abstract.
- Name the driver that is still movable before period_end: 31% seat activation at term-week 6 can be changed this term, whereas last term's usage cannot, so that is where the conversation goes.
- State what would move the score and by when, so the number arrives attached to an action rather than as a verdict.
- For the written line, use a band and a review date rather than a bare decimal, because a single number in a document will be re-quoted without its interval.
Follow-up
- The VP asks for every account scored above 0.50. What do you tell them about where that threshold came from?
- How would you check whether 0.62 is calibrated at all, and on what sample?
State the impact of your last project without inflating it
Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.
Approach
- Name the metric by its definition rather than its label. 'Verified mastery rate' means nothing without its numerator, denominator, window, and the reporting lag the delayed retention check forces.
- Separate the three claims usually merged into one: the metric moved, your work moved it, and the movement was worth what it cost.
- State the counterfactual explicitly and say what produced it: a holdout, matched sections, or a pre-period trend. If it was a before-and-after with no control, say that in those words.
- List the co-occurring changes with dates: a term boundary, a roster sync at scale, a pricing change, another team shipping into the same surface. In this domain the academic calendar alone routinely dwarfs a product effect.
- Convert the shortfall into a specific design artefact: the holdout you would have reserved, the instrumentation you would have shipped before the change, the analysis plan you would have written first.
Follow-up
- What fraction of the observed movement would you defend under cross-examination, and on what evidence?
- Your project shipped in term-week one. How does that change what you can claim?
- If the effect is genuinely not separable, was the project worth doing?
- 01
How do you handle sample ratio mismatch (SRM) when you discover it in an ongoing educational experiment?
- 02
Your renewal-risk score joins fct_subscription_period to seat activation from dim_learner.first_activity_at and to pace adherence from fct_enrollment. For one institution with 900 seats_provisioned and 31% activation at term-week 6, it outputs a 0.62 probability of non-renewal. A sales VP asks whether you are losing the account or not. You have two minutes and no slides. Deliverable: the spoken answer, plus the one line you would put in the weekly account review so the number is not later quoted as a certainty. Probed: whether you can carry uncertainty and still give a decision.
- 03
Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.
Is this an official Khan Academy interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Khan Academy. Rounds and questions reflect what candidates have reported, not a process Khan Academy has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult are the technical interviews at Khan Academy?
The interviews are moderately to highly rigorous, focusing heavily on practical application rather than trick questions. Interviewers expect clean SQL code, solid statistical reasoning, and clear structured thinking when tackling ambiguous product scenarios.
PracHub interview research ↗What is the typical timeline from the initial application to a final offer?
The process typically spans several weeks to a couple of months, depending on scheduling and team capacity. It includes an initial recruiter screen, a technical take-home project or screening call, followed by a series of deep-dive and cross-functional interviews.
PracHub interview research ↗How much emphasis is placed on the mission during the interview loops?
Mission alignment is paramount. Khan Academy operates with education at its core, and interviewers look for candidates who genuinely care about student outcomes, educational equity, and the nonprofit's broader societal impact.
PracHub interview research ↗Are there remote work options available for this role?
Yes, Khan Academy embraces a remote-first culture that caters to specific time zones, specifically supporting candidates located in the continental United States, Hawaii, and Canada.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Yale University Data Scientist Interview Questions & Guide 2026
Timed practice for Yale University data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
Universitätsklinikum Tübingen Data Scientist Interview Questions & Guide 2026
A prep plan for Universitätsklinikum Tübingen data scientist candidates built on spaced review: write a full answer before opening any solution, keep a running list of the assumptions you failed to state, and repeat every missed exercise th
University of Michigan Data Scientist Interview Questions & Guide 2026
Metric-diagnosis practice for University of Michigan data scientist candidates: a number moves week over week and you work out why, splitting by segment, ruling out a logging or release change, and separating a mix shift from a real change
University of Florida Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for University of Florida data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.