Duolingo Data Scientist Interview: A Retention and Growth Practice Case

For a Duolingo data scientist interview, a useful retention case begins with a precise question: which learners are moving between activity states, and what change might help them return? A growth forecast becomes meaningful only after the user population, time window, and transition denominators are clear.

Practice that opening with PracHub's early engagement question. Then work through the original case below: reconcile a small learning-product population, compare two proposed improvements, and explain why a promising forecast still needs an experiment.

Evidence boundary: Duolingo's public growth-model article provides historical company context. The weekly states, datasets, forecasts, and product proposals here are original hypothetical exercises. They are not Duolingo interview questions, current internal metrics, or evidence of a fixed interview sequence.

Separate company context from interview evidence

Official historical context: Duolingo's 2023 growth-model article explains how its team decomposed daily activity into mutually exclusive learner states and monitored transitions. It also describes using simulations to identify promising metrics, then experiments to investigate whether changing those metrics improved outcomes. Duolingo growth model

That publication is valuable preparation material. It is not proof that an interviewer will ask you to reproduce the model, or that the same framework describes every team's work today. This guide does not use unverified third-party round counts, timers, or workload percentages.

The practice case deliberately uses three weekly states instead of copying the historical daily model. Its purpose is to make classification and arithmetic inspectable. Do not label the resulting weekly active-user count as DAU, and do not call the case's retention parameter Duolingo's official CURR.

Define the population before counting activity

Imagine a learning product with 1,000 established learners at the end of Sunday, August 9, 2026. Every learner in this starting population has completed at least one lesson historically. For this exercise, activity means at least one valid lesson completion during a calendar week in UTC.

Week 0 runs August 3–9. Week -1 runs July 27–August 2. Classify each established learner once, using the most recent valid completion known by the snapshot cutoff:

• A: Active. Completed a lesson in Week 0.
• R: Recently inactive. No completion in Week 0, but at least one in Week -1.
• D: Longer inactive. No completion in either week, with an earlier completion in their history.

The letters are shorthand for this exercise. Registered accounts that have never completed a lesson need a separate population definition; they cannot silently become returning learners. Assume 50 people complete their first-ever lesson in Week 1 and enter the modeled population then.

Deduplicate event retries before building user activity. Resolve user identity consistently across devices. Freeze an event-time cutoff and a policy for late-arriving data. Otherwise, a learner could appear new on one device and returning on another, or historical states could change between two extracts.

Check the date boundaries on a small ledger

The following original sample illustrates classification, not the full 1,000-person population. Dates refer to each learner's latest valid completion by the cutoff.

A completion dated August 10 must not enter the August 9 snapshot. If the dataset includes it, filter it before taking the latest valid completion; an existing learner may still have an earlier qualifying event. An account with no historical completion needs the separate never-activated treatment described above.

This small Python boundary check uses standard date objects. The input is already reduced to one eligible historical date per established learner; it is not a substitute for event deduplication or identity resolution. Python date documentation

from datetime import date

def state(last_completion):
    last = date.fromisoformat(last_completion)
    assert last <= date(2026, 8, 9)
    if last >= date(2026, 8, 3):
        return "A"
    if last >= date(2026, 7, 27):
        return "R"
    return "D"

assert state("2026-08-03") == "A"
assert state("2026-08-02") == "R"
assert state("2026-07-26") == "D"

The boundary checks were executed. In an interview, explain the data assumptions before presenting a compact function like this. A short implementation can be correct for its input contract while remaining incomplete as a production pipeline.

Reconcile the starting population and transitions

Assume the Week 0 counts are 400 active, 300 recently inactive, and 300 longer inactive learners. For the baseline forecast, make these hypothetical assumptions: 70% of A complete a lesson in Week 1, 20% of R return, and 5% of D return.

A learner active in Week 1 enters A regardless of the previous state. A Week 0 active learner who does not return enters R. A learner already in R who remains inactive moves to D. A learner in D who remains inactive stays there.

Each row reconciles to its starting population. The zeros follow from the weekly definitions, not from rounding. For example, a recently inactive learner cannot remain in R after another entirely inactive week: their latest lesson is now too old.

The baseline forecast is therefore 355 active established learners. Add the separate inflow of 50 first-time learners to obtain 405 weekly active learners. The final modeled population is 405 + 120 + 525 = 1,050.

Do not add all 1,000 established learners to the active count, and do not include the new inflow in the established-user retention denominator. The case forecasts activity for a population with a defined entry event, not all registered accounts.

Compare retention and return scenarios fairly

Now consider two hypothetical proposals. A lesson-resume improvement might raise the A-to-A probability from 70% to 72%. A return reminder might raise the R-to-A probability from 20% to 24%. Leave the D return probability and the new-learner inflow unchanged for this first comparison.

The resume scenario adds 400 × 0.02 = 8 expected active learners. The reminder scenario adds 300 × 0.04 = 12. Applying both parameter changes adds 20 in this one-step model because they affect disjoint starting groups.

These are expected counts under assumptions, not observed experimental lifts. The reminder's larger modeled gain does not establish that it is easier to build, cheaper, less intrusive, or more likely to work. Compare achievable effect ranges and implementation costs before assigning priority.

Also distinguish percentage points from relative percentages. Raising 70% by two percentage points gives 72%. Raising it by 2% relatively gives 71.4%, adding only 5.6 expected active learners here. Fractional expected counts are normal in forecasts; an observed dataset still contains whole people.

Verify the arithmetic without hiding the model

A compact calculation makes the assumptions easy to change:

start = {"A": 400, "R": 300, "D": 300}

def active_next(a=0.70, r=0.20, d=0.05, new=50):
    return start["A"] * a + start["R"] * r + start["D"] * d + new

assert abs(active_next() - 405) < 1e-9
assert abs(active_next(a=0.72) - 413) < 1e-9
assert abs(active_next(r=0.24) - 417) < 1e-9
assert abs(active_next(a=0.72, r=0.24) - 425) < 1e-9

The full verification also checked that transition probabilities stay between zero and one, each row sums to one, and total people are conserved after adding new arrivals. All displayed scenario results matched those checks.

For a longer forecast, track how today's state changes alter tomorrow's population. Simply multiplying the one-week gain by 52 ignores compounding, state composition, seasonality, and changing return probabilities. The broad D category may combine learners inactive for very different lengths of time, making a constant return rate especially fragile.

Ask for historical backtests before trusting a planning forecast. Fit assumptions on an earlier period, predict a later period, and inspect error by starting state and relevant learner segments. A good aggregate fit can still conceal a poor forecast for new learners or a particular course.

Turn a promising scenario into an experiment

Official context: Duolingo's public experimentation article describes comparing product variants and considering multiple outcomes, including cases where a revenue improvement conflicted with retention. It supports treating product decisions as more than a single-metric calculation. It does not supply the design or success thresholds for this fictional reminder. Duolingo experimentation overview

For the reminder proposal, define eligibility from the pre-assignment R population. Randomize eligible learners consistently at the user level, subject to valid notification permissions. Keep the assignment stable and analyze all assigned eligible learners for the primary intent-to-treat comparison, including people who never open the reminder.

A suitable primary outcome for this case is a valid lesson completion during the prespecified following week. Its denominator is assigned eligible learners, not reminder openers. Comparing only openers would condition on behavior that the treatment may affect and select a different population.

Before launch, choose an analysis horizon, meaningful effect size, uncertainty method, and sample-size calculation based on the eligible population. The toy count of 300 recently inactive learners is an arithmetic input, not a recommendation that 300 people provide adequate power.

Include guardrails for notification opt-outs, complaints, crashes, and a learning-quality measure appropriate to the feature. Define how repeated exposure is handled. If the intervention creates social spillovers or household interactions, reconsider whether independent user assignment adequately describes the experiment.

Explain the decision when the results are mixed

Suppose an eventual experiment increases next-week lesson completions but also raises notification opt-outs beyond a prespecified acceptable boundary. A defensible recommendation could be to withhold broad rollout, inspect the affected segments, and test a less intrusive version. The original forecast does not cancel the guardrail concern.

If the confidence interval includes both a meaningful benefit and a meaningful harm, describe the result as uncertain. Do not translate “not statistically significant” into proof of no effect. Decide whether another experiment is worthwhile using expected learning value, available traffic, and product cost.

Even a clear activity increase does not automatically demonstrate better learning. A learner might complete an easy lesson solely to dismiss a reminder. Explain what additional evidence would connect the behavioral metric to durable learning value, and separate that longer-term question from the short-term return outcome.

A concise case conclusion might be: “The reminder has the larger one-week gain under our assumptions, but those assumptions are untested. I would first validate the state ledger, then run a user-randomized test with a fixed eligible denominator and learning and notification guardrails.” That is a preparation answer, not a forecast about Duolingo's actual roadmap.

Practice five connected data science questions

These are general PracHub exercises selected for this case, not a verified Duolingo interview bank.

Return to the early engagement exercise and change one definition: count a lesson start instead of a completion. Explain which state counts could change, why the forecast would need recalibration, and what behavior the new metric might reward.

Sources and Further Reading

• Duolingo: Our growth model — historical 2023 company framework, distinct from this weekly exercise.
• Duolingo: Improving Duolingo, one experiment at a time — official experimentation context.
• Python: datetime documentation — date handling used in the boundary checks.