As a Data Scientist at NYC Data Science Academy, you occupy a pivotal role that directly contributes to our mission of harnessing data to drive innovation and strategic decision-making. This position is fundamental in translating complex data into actionable insights that inform product development, enhance user experience, and ultimately influence the academy's growth trajectory. You will work at the intersection of data analysis, machine learning, and business strategy, ensuring that our initiatives are backed by robust data-driven insights.
Your work will impact various products and services, from optimizing our educational programs to developing predictive models that enhance learner engagement. You will be part of a collaborative environment, working closely with cross-functional teams, including software engineers, product managers, and educators. This role is not just about crunching numbers; it involves critical thinking, creativity, and the ability to communicate your findings effectively to diverse stakeholders. Expect to face complex challenges that require innovative solutions, making this a dynamic and rewarding position.
Initial Screening
reportedMost candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.
What to demonstrate
- Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
- Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
- The substance of the questions you ask back, which an experienced screener reads as a level signal
How to prepare
- Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
- Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
- Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered
Technical Interviews
reportedA handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.
What to demonstrate
- Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
- Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
- Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.
How to prepare
- Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
- Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
- If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows
Behavioral Assessments
reportedMost of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.
What to demonstrate
- Whether you can state the other side's argument accurately before you explain why you disagreed
- What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
- Whether you distinguish being overruled from being wrong, and can give an example of each
How to prepare
- Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
- For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
- Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.
Panel Interviews
reportedA day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.
What to demonstrate
- Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
- Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
- Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method
How to prepare
- Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
- Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
- Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
- Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times
PracHub editorial advice for the preparation topics above.
Adaptive item selection holds observed accuracy flat by construction.
A selector targeting a fixed success probability, say 70%, will keep measured first-attempt accuracy near 70% whether learners are improving or not, because as theta rises the engine simply serves harder items. Reporting 'accuracy improved 3 points' on adaptive content therefore usually means the selector got more conservative, not that anyone learned more. Accuracy is only interpretable on a fixed form with unchanged item versions, which is why the first-attempt accuracy metric above carries both restrictions.
The academic calendar creates structural breaks that look like product effects.
Term start, exam weeks, holidays, and summer each shift usage by amounts far larger than any feature change. A launch timed to week one of a term will show a large lift that is entirely calendar, and a launch in the last week of term will show a collapse. Comparisons must be term-week aligned through dim_term, and any pre/post analysis over a term boundary needs a comparison group living on the same calendar.
Reaching for a model before the target metric exists
Before naming an algorithm, write down the label, the prediction time, and the action that changes when the score crosses a threshold. If you cannot say what decision the output drives, any modelling choice is guesswork dressed up as method.
Dropping rows with missing values without naming the mechanism
Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Can you explain the concept of overfitting and how to prevent it?
Can you explain the concept of overfitting and how to prevent it?
Approach
- Set a baseline first, so any model has something honest to beat.
- Say how the offline result would be validated online before it is trusted.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- Where could label leakage enter this setup?
- How would you choose the decision threshold, and who owns that choice?
What metrics would you use to evaluate the performance of a classifica…
What metrics would you use to evaluate the performance of a classification model?
Approach
- Say how the offline result would be validated online before it is trusted.
- Check what information would not exist at prediction time, and exclude it.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
Compute pace adherence at term-week six
Given fct_enrollment (enrollment_id, learner_id, course_id, section_id, term_id, enrollment_source, enrolled_at, scheduled_start_date, due_date, completed_at, units_total, units_completed, status) and dim_term (term_id, term_start_date, term_end_date, term_length_weeks), compute the share of enrollments at or ahead of pace at the end of term-week 6. An enrollment is on pace when units_completed >= units_total * (6 / term_length_weeks). Count only enrollments active at that moment. Handle NULL scheduled_start_date and units_total of zero explicitly, and return one value per term with its denominator.
Approach
- Join to dim_term on term_id and derive the cutoff as term_start_date plus 42 days. The metric is anchored to the term calendar, not to each enrollment's own start, which is what makes it comparable across institutions on different calendars.
- Resolve NULL scheduled_start_date by falling back to term_start_date, and count how often the fallback fired. A large fallback share is a finding about roster sync, not a detail to bury.
- Reconstruct active-at-cutoff rather than trusting the status column: enrolled_at <= cutoff and (completed_at is null or completed_at > cutoff). If status is current-state only, say so, because then filtering on it removes learners who fell behind and dropped later.
- Exclude units_total = 0 from both numerator and denominator and report that count separately. Left in, 0 >= 0 marks every empty enrollment as on pace and drags the rate up.
- Group by term_id and return numerator, denominator, and rate together, never the rate alone.
Worked solution 20 min
- Merge fct_enrollment to dim_term on term_id and compute cutoff = term_start_date + pd.Timedelta(days=42).
- Build active_at_cutoff = (enrolled_at <= cutoff) & (completed_at.isna() | (completed_at > cutoff)), and record how many rows the status column alone would have excluded.
- threshold = units_total * 6 / term_length_weeks; on_pace = units_completed >= threshold, evaluated only where units_total > 0.
- Group by term_id, aggregate on_pace.sum() and the eligible row count, and attach the NULL-start fallback count and the zero-units count as separate columns.
Follow-up
- A two-week holiday falls inside weeks one to six for one term. How do you stop that punishing those enrollments without hand-editing the threshold?
- Two terms have term_length_weeks of 10 and 16. Is a single week-6 number comparable across them, and what would you report instead?
Daily scored submissions under rubric scoring lag
fct_assessment_response(response_id, learner_id, content_item_id, content_version_no, submitted_at, is_correct, score_points, max_points, scoring_mode, scored_at) stores both timestamps in UTC. scored_at is NULL until a rubric item is graded and lags submitted_at by days when scoring_mode = 'rubric_human'. Produce two dense daily series over the last 90 days: submissions keyed on submission date, and completed scorings keyed on scored date. Then state the reporting lag after which each series stops moving, computed from the data rather than asserted, and say what biases your lag estimate.
Approach
- Key the two series on different columns on purpose: date(submitted_at) for volume, date(scored_at) for graded throughput, with scored_at IS NOT NULL on the second. They answer different questions and are not two versions of one number.
- Define scored as scored_at IS NOT NULL rather than is_correct IS NOT NULL. Check first whether rubric rows carry score_points with is_correct left NULL after grading; if they do, the is_correct test undercounts by exactly the rubric share.
- Estimate lag empirically with percentiles of scored_at - submitted_at grouped by scoring_mode. This is computed only over rows already scored, so it is survivorship-biased downward. Report alongside it the share of submissions in each age bucket still unscored.
- Generate a dense date spine and LEFT JOIN both series onto it so a zero-volume day is a 0, not an absent row that a chart will interpolate across.
- Name the timezone problem instead of guessing: these tables carry a locale on the learner but no timezone, so a local-calendar day series is not computable from this schema. Say that rather than pretending date(submitted_at) is an institution's day.
Worked solution 15 min
- Build the date spine for the 90-day window, then the two aggregates, then join.
- Run a distribution of is_correct for rows with scored_at IS NOT NULL to decide which column defines scored.
- Compute median and p95 of scored_at - submitted_at per scoring_mode, and the unscored share by 1, 3, 7 and 14 day age buckets.
- Write one sentence stating the reporting lag you are imposing and what it costs.
Follow-up
- The last five days of the submitted series will be revised upward as late rows land. How do you present a series whose recent points are known to be incomplete?
- Rubric grading is done by people with weekly schedules. What does that do to a daily scored series, and would an ISO week grain be more honest?
- If you had to pick a single cut-off date after which the data is considered final, how would you derive it and what percentage of eventual scorings would you be discarding?
Rebuild sessions and active seconds from heartbeat events
Given raw_heartbeat(learner_id, content_item_id, content_version_no, heartbeat_at TIMESTAMP UTC, session_hint VARCHAR NULL), reconstruct the grain of fct_lesson_activity: one row per learner per content item per session. Close a session after 30 minutes with no heartbeat. Compute active_seconds as the sum of inter-heartbeat gaps inside a session with each gap capped at 120 seconds, and wall_seconds as last minus first heartbeat. Ignore session_hint, which the client sets unreliably. Then reconcile your active_seconds against the stored column in fct_lesson_activity and explain any systematic difference.
Approach
- Compute prev_at = LAG(heartbeat_at) OVER (PARTITION BY learner_id, content_item_id, content_version_no ORDER BY heartbeat_at) and gap_seconds from the difference. Partition on the version too, since the same item at a new version is different content.
- Flag is_new_session = (prev_at IS NULL OR gap_seconds > 1800), then session_seq = SUM(is_new_session::int) OVER (same partition ORDER BY heartbeat_at ROWS UNBOUNDED PRECEDING). The running sum over an ordered flag is the island assignment and it stays correct at any data volume, unlike a self-join on nearby timestamps.
- Aggregate per (partition, session_seq): active_seconds = SUM(CASE WHEN is_new_session THEN 0 ELSE LEAST(gap_seconds, 120) END), wall_seconds = MAX(heartbeat_at) - MIN(heartbeat_at), plus heartbeat_count.
- State the two consequences of the definition before anyone asks: a single-heartbeat session has active_seconds = 0 and wall_seconds = 0, and the per-gap cap means active_seconds can never exceed 120 * (heartbeat_count - 1), so a client with a slow heartbeat interval understates real time.
- Reconcile by joining on (learner_id, content_item_id, started_at) and examining the distribution of the difference, not a row count. A uniform shift points at the cap or the close rule; a heavy one-sided tail points at heartbeats the stored column excludes.
Follow-up
- The stored column excludes backgrounded tabs but raw_heartbeat has no visibility flag. How would you detect backgrounded stretches from timing alone, and how confident should you be?
- Two heartbeats can share a timestamp. What does that do to LAG, and does it change the answer?
- A learner studies for two hours with a 40-minute dinner break in the middle. Your rule gives two sessions. Which downstream metrics care about that choice and which do not?
Given a dataset with multiple features, how would you determine which …
Given a dataset with multiple features, how would you determine which features are most important?
Approach
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How would you approach a project where the data is insufficient to dra…
How would you approach a project where the data is insufficient to draw conclusions?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How would you approach analyzing a drop in user engagement for an onli…
How would you approach analyzing a drop in user engagement for an online course?
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Fix the population and the time window before naming any metric.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
How do you prioritize your work when faced with multiple deadlines?
How do you prioritize your work when faced with multiple deadlines?
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
Explain the difference between supervised and unsupervised learning.
Explain the difference between supervised and unsupervised learning.
Approach
- Work from the decision backwards to the evidence you would need.
- Clarify what is being asked and what a complete answer would contain.
- Say what you would check first and why it is the highest-information step.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Apply CUPED and stratification to an under-powered clustered test
A section-randomised test is under-powered and the team cannot get more sections. Eight term-weeks of pre-period fct_assessment_response and fct_lesson_activity exist before assignment. The learner-level correlation between pre-period and in-test weekly mastery events is 0.55; at section-mean level it is 0.72. Propose a variance-reduction plan. Compute the MDE multiplier CUPED buys at each level, say which level you apply it at and why, and handle the learners with no pre-period because roster_sync created their accounts after assignment.
Approach
- State the estimator: Y_adj = Y - theta x (X - mean(X)), where X is the pre-period covariate and the binding precondition is that X is measured strictly before assignment. Fit theta = Cov(Y, X) / Var(X) on the in-test data pooled across both arms. That is standard CUPED and it is the only option here: Y is the in-test outcome, so theta cannot be computed from pre-period data alone. Pooling is what keeps it honest, because a pre-assignment X has the same distribution in both arms and the adjustment therefore cannot absorb the treatment effect. Var(Y_adj) = (1 - r^2) x Var(Y), so the MDE scales by sqrt(1 - r^2).
- Apply it at the unit of analysis. Under section randomisation that is the section mean, so the 0.72 section-level correlation is the one that matters. It exceeds the learner-level 0.55 because averaging removes idiosyncratic learner noise while preserving the section's persistent level.
- Stratify the randomisation itself rather than only adjusting afterwards: block sections on pre-period mastery-event quartile crossed with grade_band, then randomise within block. With 300 sections per arm, blocking also protects against a bad draw, which post-hoc CUPED cannot undo.
- Handle the no-pre-period learners without imputing a value that would drag theta. Place them in their own stratum with X set to that stratum's mean, which leaves them with zero adjustment and no bias, and report their share alongside the result.
- Hold the two rules that actually protect the estimate: the covariate must be measured strictly before assignment, and theta must be fitted once on the pooled arms rather than separately within each arm. A post-assignment covariate can itself be moved by treatment, so adjusting on it subtracts part of the effect you are measuring. Arm-specific thetas make the adjustment differ by arm, which reintroduces the same bias by another route. Fitting theta on the data it adjusts carries a finite-sample bias of order 1 / n_clusters; at 300 sections per arm that is negligible, and if you want it gone entirely, estimate theta from an earlier pre-period regression on data outside the test window and accept a slightly lower r.
Worked solution 30 min
- Learner level: sqrt(1 - 0.55^2) = sqrt(0.6975) = 0.835, a 16.5% smaller MDE.
- Section level: sqrt(1 - 0.72^2) = sqrt(0.4816) = 0.694, a 30.6% smaller MDE.
- Convert to sample equivalence: 1 / 0.4816 = 2.08, so section-level CUPED is worth roughly doubling the number of sections.
- Build the blocks as four pre-period quartiles crossed with grade_band, randomise within block, and persist the block id for the analysis.
- Fit theta once on the pooled section-level in-test data, form Y_adj, then estimate the effect at section level with block fixed effects and compare the adjusted and unadjusted estimates and intervals.
Follow-up
- Your pre-period window spans a holiday for half the sections. What does that do to the correlation and to theta?
- Can you stack CUPED on top of stratification, and does the second one still buy anything once the first has run?
- The section-level correlation is 0.72 overall but 0.45 for sections whose instructor changed between periods. How do you handle that subgroup?
Active minutes fell eighteen percent with session count flat
Weekly active learning minutes per active learner fell from 42 to 34 the week a new client build shipped. Distinct learner_id and row count in fct_lesson_activity are both flat week over week. You have fct_lesson_activity (session_id, started_at, ended_at, active_seconds, wall_seconds, device_type, content_version_no, activity_date) and nothing else today. Decide whether this is a real engagement change or an instrumentation change, name the column that settles it, and write the query. Deliverable: a one-paragraph call within the day.
Approach
- Compute the daily median of active_seconds / wall_seconds per session for fourteen days either side of the release. active_seconds is heartbeat-derived and capped per gap; wall_seconds is ended_at minus started_at and is untouched by heartbeat cadence, so the ratio isolates the instrument from the behaviour.
- Hold the denominator fixed alongside it: sessions per learner per week and median wall_seconds per session. If both are flat, nothing about how long learners stayed changed, which leaves only how time was counted.
- Split the ratio by device_type. Managed Chromebook fleets typically run a pinned build and update on a school IT cycle, so that group is a natural unexposed comparison for a client-side change.
- State the mechanism explicitly: active_seconds sums inter-heartbeat gaps with each gap capped at 120 seconds, so a heartbeat interval raised above 120 seconds discards the excess on every gap and shrinks active_seconds proportionally while wall_seconds is unaffected.
- Size it: apply the pre-release ratio to post-release wall_seconds to get counterfactual minutes, and report the difference from the published number as the artefact.
Follow-up
- What would you backfill, and what would you leave as a documented break in the series?
- If wall_seconds had fallen by the same proportion, what would you check next before calling the drop real?
- How would you make this metric robust to the next heartbeat cadence change without re-litigating it?
For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Design one test end to end on paper
- Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
- Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
- State in advance what you will do if the primary metric is flat while a secondary metric is significant.
Deliverable: A one-page test design with a decision rule written before launch.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02Power arithmetic until it is automatic
- Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
- Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
- Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.
Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.
Practice prompt ↗Practice prompt ↗03Variance and the unit-of-analysis problem
- Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
- Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
- Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.
Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.
Practice prompt ↗Practice prompt ↗04Validity threats you can actually test for
- Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
- Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
- Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.
Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.
Practice prompt ↗Practice prompt ↗Worked solution ↗05When randomization is not available
- Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
- Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
- List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.
Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.
Practice prompt ↗Practice prompt ↗06The readout query
- Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
- Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
- Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.
Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.
Practice prompt ↗Practice prompt ↗07Present it to someone who will not read the appendix
- Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
- Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
- Rewrite your opening line so the recommendation lands before any methodology.
Deliverable: A one-page readout whose first line is the recommendation.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.
Describe a time you encountered a significant challenge in a project a…
Describe a time you encountered a significant challenge in a project and how you resolved it.
Approach
- Name the disagreement or constraint, and how you resolved it with evidence.
- State the situation in two sentences and spend the rest on your reasoning.
- Close with what you would do differently, concretely.
Follow-up
- How did you know the outcome was caused by your change?
- What did you decide not to do, and why?
Describe a situation where you had to deal with conflict within a team…
Describe a situation where you had to deal with conflict within a team.
Approach
- Name the disagreement or constraint, and how you resolved it with evidence.
- State the situation in two sentences and spend the rest on your reasoning.
- Quantify the outcome, including what you would not claim credit for.
Follow-up
- What did you decide not to do, and why?
- How did you know the outcome was caused by your change?
State the impact of your last project without inflating it
Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.
Approach
- Name the metric by its definition rather than its label. 'Verified mastery rate' means nothing without its numerator, denominator, window, and the reporting lag the delayed retention check forces.
- Separate the three claims usually merged into one: the metric moved, your work moved it, and the movement was worth what it cost.
- State the counterfactual explicitly and say what produced it: a holdout, matched sections, or a pre-period trend. If it was a before-and-after with no control, say that in those words.
- List the co-occurring changes with dates: a term boundary, a roster sync at scale, a pricing change, another team shipping into the same surface. In this domain the academic calendar alone routinely dwarfs a product effect.
- Convert the shortfall into a specific design artefact: the holdout you would have reserved, the instrumentation you would have shipped before the change, the analysis plan you would have written first.
Follow-up
- What fraction of the observed movement would you defend under cross-examination, and on what evidence?
- Your project shipped in term-week one. How does that change what you can claim?
- If the effect is genuinely not separable, was the project worth doing?
- 01
Describe a time you encountered a significant challenge in a project and how you resolved it.
- 02
Describe a situation where you had to deal with conflict within a team.
- 03
Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.
Is this an official NYC Data Science Academy interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at NYC Data Science Academy. Rounds and questions reflect what candidates have reported, not a process NYC Data Science Academy has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗What is the typical interview difficulty and preparation time?
The interview process is considered to be of average difficulty, focusing on both technical and behavioral aspects. Candidates typically spend 2-4 weeks preparing, depending on their current knowledge and experience.
PracHub interview research ↗What differentiates successful candidates?
Successful candidates demonstrate a strong balance of technical expertise and interpersonal skills. They effectively communicate their insights and align with the values of NYC Data Science Academy.
PracHub interview research ↗What is the culture and working style at NYC Data Science Academy?
The culture emphasizes collaboration, innovation, and a user-focused approach to data science. Team members are encouraged to share ideas and work together to solve complex problems.
PracHub interview research ↗What is the typical timeline from initial screen to offer?
The timeline can vary but generally ranges from 4 to 6 weeks, depending on the scheduling of interviews and feedback collection.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Yale University Data Scientist Interview Questions & Guide 2026
Timed practice for Yale University data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
Universitätsklinikum Tübingen Data Scientist Interview Questions & Guide 2026
A prep plan for Universitätsklinikum Tübingen data scientist candidates built on spaced review: write a full answer before opening any solution, keep a running list of the assumptions you failed to state, and repeat every missed exercise th
University of Michigan Data Scientist Interview Questions & Guide 2026
Metric-diagnosis practice for University of Michigan data scientist candidates: a number moves week over week and you work out why, splitting by segment, ruling out a logging or release change, and separating a mix shift from a real change
University of Florida Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for University of Florida data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.