As a Data Scientist at Flatiron School, you will play a pivotal role in shaping the educational experiences of aspiring data professionals. This position is essential not only for the development of data science curricula but also for influencing how students engage with complex data-driven concepts. You will be working to enhance the understanding of machine learning, statistical analysis, and data interpretation among students, which directly impacts their career trajectories and the overall quality of the educational programs offered.
Your work will involve collaborating with cross-functional teams, including instructors, curriculum developers, and student support, to ensure that the learning materials are relevant and effective. You will contribute to innovative projects that leverage data to assess student outcomes, optimize learning pathways, and adapt content based on real-world applications. This role is challenging and rewarding, providing you with the opportunity to make a significant impact on the future of education in data science.
Phone Screen
reportedWhoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.
What to demonstrate
- Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
- Whether your reason for wanting the role points at the work itself rather than the company's reputation
- Whether your language signals the level being screened for: what you decided yourself versus what you were handed
How to prepare
- Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
- Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
- Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands
Technical Interviews
reportedBefore anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.
What to demonstrate
- Whether an ambiguous term becomes a specific column and filter before any computation happens
- Whether you read the schema for keys and cardinality rather than only for column names
- Whether the result answers the question at the grain it was asked at, per user or per session or per day
How to prepare
- Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
- On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
- Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over
Collaboration Assessment
reportedA handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.
What to demonstrate
- Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
- Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
- Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.
How to prepare
- Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
- Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
- If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows
PracHub editorial advice for the preparation topics above.
Learner-level standard errors on class-level interventions.
Anything an instructor controls, and anything deployed by school, is assigned at the section or school level, and outcomes within a section are correlated through the shared instructor, schedule, and device fleet. Treating the learner as the unit of independence understates variance by the design effect 1 + (m - 1) * rho for roughly equal cluster sizes. At a typical section size of 25 and rho of 0.15 that is a factor near 4.6 on variance, which routinely converts a null into a 'significant' result.
Adaptive item selection holds observed accuracy flat by construction.
A selector targeting a fixed success probability, say 70%, will keep measured first-attempt accuracy near 70% whether learners are improving or not, because as theta rises the engine simply serves harder items. Reporting 'accuracy improved 3 points' on adaptive content therefore usually means the selector got more conservative, not that anyone learned more. Accuracy is only interpretable on a fixed form with unchanged item versions, which is why the first-attempt accuracy metric above carries both restrictions.
Writing SQL without stating NULL and tie-breaking behaviour
Before calling a query finished, say what it does with NULLs, ties and empty groups. NOT IN against a subquery containing a single NULL returns no rows at all, and RANK, DENSE_RANK and ROW_NUMBER differ precisely on ties, so name which one the question requires.
Answering a product-sense question with a list of features
Answer with a decision and the measurement that would settle it: the hypothesis, the primary metric, the guardrails, and the result that would make you not ship. A feature brainstorm cannot be wrong, which is exactly why it earns no points.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Explain the concept of overfitting and how to prevent it.
Explain the concept of overfitting and how to prevent it.
Approach
- Say how the offline result would be validated online before it is trusted.
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Check what information would not exist at prediction time, and exclude it.
Follow-up
- Where could label leakage enter this setup?
- What would you monitor after launch to know the model is still valid?
What are precision and recall, and why are they important?
What are precision and recall, and why are they important?
Approach
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Check what information would not exist at prediction time, and exclude it.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
Follow-up
- Where could label leakage enter this setup?
- What would you monitor after launch to know the model is still valid?
Implement verified mastery events with a delayed check
From fct_assessment_response (learner_id, objective_id, content_item_id, submitted_at, attempt_no, is_correct, score_points, max_points, scoring_mode, scored_at, mastery_state_after), count verified mastery events for one calendar month. A mastery event is the first time a (learner_id, objective_id) pair reaches mastery_state_after 'mastered'. It is verified when the earliest scored response on that objective submitted at least 7 days later scores score_points/max_points >= 0.8. Return the monthly count and the verified rate, with the denominator restricted to events that have an eligible later check.
Approach
- Filter to rows with mastery_state_after = 'mastered', then take the earliest submitted_at per (learner_id, objective_id). This is not the same as the first row overall for the pair, since earlier attempts sit in 'learning'.
- Restrict those crossings to the target month in UTC. A pair that first crossed in an earlier month never enters this month's count even if it crosses again after a lapse, so the first-crossing filter must run over all history, not just the month.
- Build the candidate check set by joining the full response table back on (learner_id, objective_id) and keeping rows with submitted_at >= crossing + 7 days, score_points not null, scored_at not null, and max_points > 0.
- Take the earliest eligible candidate per pair, not the best one. Taking the maximum score turns a retention check into an ever-passed-later check, which is a much weaker claim and a materially higher number.
- Mark verified where score_points / max_points >= 0.8. The denominator is crossings with at least one eligible candidate; report the count of crossings with none alongside, because that share moves with the reporting lag and a large share means the month is not ready to publish.
Worked solution 30 min
- mastered = df[df.mastery_state_after == 'mastered']; crossings = mastered.groupby(['learner_id','objective_id'], as_index=False).submitted_at.min().
- month_crossings = crossings[crossings.submitted_at.between(month_start, month_end)].
- Merge the full response frame onto month_crossings on (learner_id, objective_id), keep rows where submitted_at >= crossing + 7 days and score_points.notna() and scored_at.notna() and max_points > 0.
- Sort candidates by submitted_at and take the first per pair with drop_duplicates on the pair key.
- verified = (score_points / max_points) >= 0.8; report verified.sum(), len(candidates), and len(month_crossings) - len(candidates).
Follow-up
- A pair crosses to 'mastered', lapses, and crosses again inside the same month. What does your code count, and what should it count?
- Why does this metric need a 30-day reporting lag, and what shape does the series take if you publish it at 7 days?
- The 0.8 ratio is a product choice. How would you check it actually separates learners rather than just passing almost everyone?
Daily scored submissions under rubric scoring lag
fct_assessment_response(response_id, learner_id, content_item_id, content_version_no, submitted_at, is_correct, score_points, max_points, scoring_mode, scored_at) stores both timestamps in UTC. scored_at is NULL until a rubric item is graded and lags submitted_at by days when scoring_mode = 'rubric_human'. Produce two dense daily series over the last 90 days: submissions keyed on submission date, and completed scorings keyed on scored date. Then state the reporting lag after which each series stops moving, computed from the data rather than asserted, and say what biases your lag estimate.
Approach
- Key the two series on different columns on purpose: date(submitted_at) for volume, date(scored_at) for graded throughput, with scored_at IS NOT NULL on the second. They answer different questions and are not two versions of one number.
- Define scored as scored_at IS NOT NULL rather than is_correct IS NOT NULL. Check first whether rubric rows carry score_points with is_correct left NULL after grading; if they do, the is_correct test undercounts by exactly the rubric share.
- Estimate lag empirically with percentiles of scored_at - submitted_at grouped by scoring_mode. This is computed only over rows already scored, so it is survivorship-biased downward. Report alongside it the share of submissions in each age bucket still unscored.
- Generate a dense date spine and LEFT JOIN both series onto it so a zero-volume day is a 0, not an absent row that a chart will interpolate across.
- Name the timezone problem instead of guessing: these tables carry a locale on the learner but no timezone, so a local-calendar day series is not computable from this schema. Say that rather than pretending date(submitted_at) is an institution's day.
Worked solution 15 min
- Build the date spine for the 90-day window, then the two aggregates, then join.
- Run a distribution of is_correct for rows with scored_at IS NOT NULL to decide which column defines scored.
- Compute median and p95 of scored_at - submitted_at per scoring_mode, and the unscored share by 1, 3, 7 and 14 day age buckets.
- Write one sentence stating the reporting lag you are imposing and what it costs.
Follow-up
- The last five days of the submitted series will be revised upward as late rows land. How do you present a series whose recent points are known to be incomplete?
- Rubric grading is done by people with weekly schedules. What does that do to a daily scored series, and would an ISO week grain be more honest?
- If you had to pick a single cut-off date after which the data is considered final, how would you derive it and what percentage of eventual scorings would you be discarding?
Deduplicate retried submissions before computing first-attempt accuracy
A client retry bug wrote duplicates into fct_assessment_response(response_id, learner_id, content_item_id, content_version_no, attempt_no, submitted_at, is_correct, max_points, scoring_mode): the same learner, item, version and attempt_no appears two or three times, submitted_at within two seconds, different response_id. Deduplicate keeping the earliest row per logical attempt, then compute 28-day first-attempt accuracy over items whose course has dim_course.uses_adaptive_selection = FALSE and whose (content_item_id, version_no) did not change inside the window. Report how many rows the dedup removed and its share of raw rows.
Approach
- Rank with ROW_NUMBER() OVER (PARTITION BY learner_id, content_item_id, content_version_no, attempt_no ORDER BY submitted_at, response_id) and keep rn = 1. Including response_id in the ORDER BY makes the survivor deterministic when submitted_at ties, which it will at second resolution.
- Validate the duplicate definition before deleting anything. Count partitions with rn > 1 and inspect MAX(submitted_at) - MIN(submitted_at) inside each. A spread of minutes is a genuine re-attempt the client failed to increment attempt_no for, and dropping it is data loss rather than dedup.
- Apply the version restriction by excluding content_item_id values that show more than one distinct content_version_no among responses in the window, or whose dim_content_item.published_at falls inside it. An item edited mid-window mixes two different questions into one accuracy figure.
- Filter attempt_no = 1 AND is_correct IS NOT NULL. Treating NULL is_correct as incorrect biases accuracy down by exactly the unscored rubric share, and that share is not stable week to week.
- Report accuracy with its denominator, plus removed_rows and removed_rows / raw_rows, so the size of the bug is visible next to the number it corrupted.
Follow-up
- attempt_no is 1-based within learner and item, not within item version. If an item is edited between a learner's first and second attempt, which row is the first attempt on the new version, and does the metric want it?
- The retry bug fires more often on poor connections. If that population differs in accuracy, what does silently dropping their duplicates do to the number?
- How would you make this query cheap to leave in place after the bug is fixed, so it costs nothing on a clean table?
Describe how you would conduct a statistical analysis to determine if …
Describe how you would conduct a statistical analysis to determine if a new feature is valuable.
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Fix the population and the time window before naming any metric.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
How would you assess the effectiveness of a new marketing strategy usi…
How would you assess the effectiveness of a new marketing strategy using data?
Approach
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
How do you prioritize tasks when managing multiple projects?
How do you prioritize tasks when managing multiple projects?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Fix the population and the time window before naming any metric.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
Given a dataset, how would you approach the problem of predicting cust…
Given a dataset, how would you approach the problem of predicting customer churn?
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- How would you detect that the metric is being gamed rather than genuinely improving?
How do you assess whether a student is struggling with a concept?
How do you assess whether a student is struggling with a concept?
Approach
- Work from the decision backwards to the evidence you would need.
- Say what you would check first and why it is the highest-information step.
- Clarify what is being asked and what a complete answer would contain.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Pick randomisation units for two features with different carryover
Two features need tests. Feature A shows each learner the peer explanations other learners in the same class section submitted. Feature B changes the turnaround target for rubric_model scoring from four hours to fifteen minutes, served from one shared scoring queue. Product wants both learner-randomised. Using fct_lesson_activity.is_assigned, fct_assessment_response.scoring_mode and fct_enrollment.section_id, propose a randomisation unit for each, name the interference each one suffers, and state the condition that makes a switchback design valid for one of them and invalid for the other.
Approach
- Feature A leaks by construction. A treated learner's explanation renders to control learners in the same section, so the control arm is partly treated and the estimate is attenuated toward zero by an amount you cannot recover. Randomise at section_id, or at the school when sections share an instructor or a device cart.
- Price the cluster design before agreeing to it. At section size 25 and intraclass correlation 0.15 the design effect is 1 + 24 x 0.15 = 4.6, so budget roughly 4.6 times the learner count a learner-randomised test would have needed.
- Feature B's interference runs through the shared resource, not the classroom. Serving a fifteen-minute target to half the traffic changes queue depth for the other half, so unit-level randomisation is contaminated at the queue regardless of who the units are.
- Apply the switchback condition: the treatment effect must not persist past the switch. Feature B acts on scoring latency, which resets the moment the config flips, so a time-blocked switchback over the whole queue is valid. Randomise the sequence and analyse at the block level, targeting at least 30 blocks per arm because blocks, not learners, are the unit.
- Feature A fails that condition. Seeing a peer explanation changes what a learner knows, and knowledge does not revert when the flag does, so every post-switch control block is permanently contaminated. Switchback is the wrong tool for anything that changes mastery state.
- Write a washout into Feature B's plan: discard the first block after each switch while the queue drains, and confirm the drain time from the observed scored_at minus submitted_at distribution rather than assuming it.
Worked solution 30 min
- For each feature, write the exposure mapping explicitly: whose outcome changes when one unit is treated.
- For Feature A, set the unit to section_id and compute the sample multiplier 1 + (m - 1) x rho = 4.6 at m = 25, rho = 0.15.
- For Feature B, fix a six-hour block, count the blocks available in the test window, and confirm at least 30 blocks land in each arm.
- Specify analysis up front: section-clustered standard errors for A, block-level means with a randomisation-inference p-value for B.
- Add the washout rule for B and a spillover diagnostic for A comparing control-arm outcomes in sections with high versus low treated share.
Follow-up
- For Feature B, how do you choose the switch interval, and what does shortening it cost you?
- Feature A is section-randomised, but two sections in the study share an instructor who teaches both. What do you do about that pair?
- How would you measure the size of Feature A's spillover rather than only designing around it?
Pace adherence at term-week six rose eleven points
Pace adherence at term-week 6 rose from 48 percent to 59 percent between two consecutive terms, and the dashboard query changed owners last month. Tables: fct_enrollment (enrollment_id, learner_id, course_id, units_completed, units_total, status, scheduled_start_date, due_date, enrolled_at) and dim_term (term_id, term_start, term_length_weeks). Nothing shipped to pacing in that period. Decide whether learners are pacing better, and reconcile the old and new numbers to the point.
Approach
- Diff the definitions before touching the data. Recover the prior query and compare it line by line against the current one on four axes: the denominator population, the status filter, how term-week 6 is derived, and what happens to enrollments with NULL scheduled_start_date.
- Recompute both definitions on the same term and the same enrollment set. If the difference does not account for the full eleven points there is a second cause, and knowing that before you start hunting saves the hunt.
- Test the survivorship mechanism directly: count enrollments with status in ('dropped','expired') at the end of term-week 6 and compute their units_completed over units_total. If that ratio sits well below the active median, removing them from the denominator lifts the rate mechanically and the lift is arithmetic, not behaviour.
- Verify the numerator independently, because units_completed is maintained incrementally rather than derived on read. Recount it from lesson activity for a sample of a couple of hundred enrollments and compare against the stored column to rule out a backfill as a second mover.
- Confirm term-week 6 maps to the intended dates in both terms through dim_term rather than calendar week, since an edited term_start silently shifts the evaluation window and can move the number on its own.
Follow-up
- The old definition penalises a course for enrollments that dropped in week two. Is that the behaviour you actually want from a pacing metric, and who is the metric for?
- How do you treat enrollments with NULL scheduled_start_date, and what does each choice do to the reported number?
For someone who has spent the last year in notebooks, dashboards or modelling work and has not written raw SQL under time pressure. The first four days rebuild query fluency against a fixture you control and can verify by hand; the last three attach that fluency to the rest of the loop.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Build a fixture you can check answers against
- Create a local Postgres or SQLite database with four tables (users, sessions, events, orders) holding roughly 200 rows you generated yourself, so you know the contents well enough to predict every result.
- Deliberately seed the cases that break queries: a user with no sessions, a session with no events, two orders sharing a timestamp, a NULL in one join key, and one duplicated user row.
- Before writing any SQL, hand-compute five answers on paper (how many users placed at least one order, median orders per ordering user, and three others) and save them as the ground truth for the week.
Deliverable: A one-command seed script plus a text file of five hand-computed answers to grade every later query against.
Practice prompt ↗Practice prompt ↗Practice prompt ↗Worked solution ↗02Joins, filters and NULL semantics
- Answer "which users have no orders" three ways (LEFT JOIN with IS NULL, NOT EXISTS, NOT IN) and confirm that the NOT IN version returns zero rows once the subquery contains a NULL, because the comparison is never TRUE.
- Reproduce the LEFT JOIN that silently collapses to an inner join by putting a right-table predicate in WHERE, then fix it by moving the predicate into the ON clause, and record both row counts.
- Create a fan-out bug on purpose by joining orders to order_items and summing the order total, then correct it with a pre-aggregated subquery and explain in one line which table changed the grain.
Deliverable: One annotated .sql file holding the three join traps, each with the wrong result and the corrected result side by side.
Practice prompt ↗Practice prompt ↗03Window functions and frames
- Write three window queries against the fixture: a running order total per user, the rank of each order within its user by value, and the day gap to that user's previous order, then check each against the day-one ground truth.
- Run ROW_NUMBER, RANK and DENSE_RANK over a column containing ties, print all three side by side, and write one sentence on when each is the correct choice.
- Switch one query from the default frame (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, which is what you get when ORDER BY is present and no frame is written) to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and explain why the output differs only when the ORDER BY column has duplicates.
Deliverable: Three verified window queries plus a short note explaining the RANGE versus ROWS difference in your own words.
Practice prompt ↗Practice prompt ↗04The four analytical query patterns
- Write a monthly retention grid: first order month per user, then months-since-first as the column, and verify that month zero equals the cohort size exactly.
- Sessionize the events table under a 30-minute inactivity rule using LAG plus a cumulative sum over a new-session flag.
- Build a four-step funnel that counts distinct users rather than events at each step, and state the rule you applied to a user who reaches step three without ever logging step two.
Deliverable: One file with the retention, sessionization and funnel patterns, each carrying a one-line note on the assumption it bakes in.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Write SQL the way you will have to write it live
- Set a 12-minute timer and solve three medium prompts in a plain editor with no execution and no autocomplete, then run them and tally syntax errors separately from logic errors.
- Narrate one solution aloud while writing it, stating the grain of each intermediate result (one row per user, one row per user-day) before you type its body.
- Rewrite your slowest solution as a CTE chain where every CTE name states its grain, and time yourself re-solving it from blank.
Deliverable: A recording of one narrated solution plus an error tally that separates syntax from logic.
Practice prompt ↗Practice prompt ↗06One day for everything that is not SQL
- Write the preconditions of the two-sample t-test from memory, then check them: independent observations, and a difference in means whose sampling distribution is approximately normal, which at large sample sizes follows from the central limit theorem rather than from normality of the raw values.
- Write the difference between an odds ratio from logistic regression and a relative risk, and state the condition under which the two are close (low outcome prevalence).
- Prepare a 90-second answer to "how would you know this model is any good" that names the metric, the baseline you would beat, and the cost of the errors you care about.
Deliverable: One page of notes covering test preconditions, the odds-ratio caveat and the model-quality answer.
Practice prompt ↗Practice prompt ↗07Full loop rehearsal
- Run a 45-minute mock with someone willing to interrupt: 20 minutes of SQL, 15 minutes defining a metric, 10 minutes on a past project.
- Re-solve from blank the two queries you were slowest on this week and compare the times against day five.
- Write a five-line answer to "walk me through a project" that puts a number in the first sentence and names the decision the work changed.
Deliverable: Mock feedback notes plus a timed project narrative you can deliver without reading it.
Practice prompt ↗Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.
What motivates you to work in the field of data science?
What motivates you to work in the field of data science?
Approach
- Quantify the outcome, including what you would not claim credit for.
- State the situation in two sentences and spend the rest on your reasoning.
- Pick a story where you drove the decision, not one where you observed it.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Describe a challenge you faced in a previous role and how you overcame…
Describe a challenge you faced in a previous role and how you overcame it.
Approach
- Quantify the outcome, including what you would not claim credit for.
- Pick a story where you drove the decision, not one where you observed it.
- Close with what you would do differently, concretely.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Disagree with a product manager using evidence, not volume
A product manager wants to ship an adaptive practice selector to all grade bands on the strength of a four-point rise in first-attempt accuracy during a six-week pilot. You believe the rise is an artefact of the selector's target success rate. You have fct_assessment_response, dim_content_item with irt_a, irt_b and calibration_n, and the pilot arm assignment. Deliverable: the analysis that tests your objection, and how you present it so the PM can change position without it reading as a defeat. Probed: whether you make disagreement falsifiable rather than rhetorical.
Approach
- Make the objection falsifiable before raising it. The claim implies two testable predictions: mean calibrated difficulty of served items rose with learner ability, and accuracy is flat within ability strata.
- Compute weekly mean irt_b of served items per arm, restricted to items with calibration_n above your floor, and plot it against the accuracy series. If served difficulty tracked ability, the accuracy line carries no learning signal and you can show that rather than assert it.
- Build the metric that survives adaptivity: a small fixed-form set with (content_item_id, version_no) held constant, served to both arms, reported as the pilot's accuracy readout.
- Bring the replacement to the meeting, not only the refutation. A PM who has been told the number is meaningless still has a launch decision and no instrument.
- Separate the two questions out loud: whether the selector helps learners is open and testable; whether first-attempt accuracy measures it is settled, and it does not.
Follow-up
- The fixed-form set costs each learner six minutes a fortnight. How do you justify that to the same PM?
- Mean served irt_b is flat but accuracy still rose four points. What do you look at next?
- 01
What motivates you to work in the field of data science?
- 02
Describe a challenge you faced in a previous role and how you overcame it.
- 03
A product manager wants to ship an adaptive practice selector to all grade bands on the strength of a four-point rise in first-attempt accuracy during a six-week pilot. You believe the rise is an artefact of the selector's target success rate. You have fct_assessment_response, dim_content_item with irt_a, irt_b and calibration_n, and the pilot arm assignment. Deliverable: the analysis that tests your objection, and how you present it so the PM can change position without it reading as a defeat. Probed: whether you make disagreement falsifiable rather than rhetorical.
Is this an official Flatiron School interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Flatiron School. Rounds and questions reflect what candidates have reported, not a process Flatiron School has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult is the interview process for this role?
The interview process is moderate in difficulty, requiring a solid understanding of data science concepts and the ability to communicate effectively. Expect a mix of technical and behavioral interviews.
PracHub interview research ↗What separates successful candidates from others?
Successful candidates demonstrate both technical expertise and a genuine passion for teaching and mentoring in data science. Effective communication and collaboration skills are equally important.
PracHub interview research ↗What is the culture like at Flatiron School?
The culture at Flatiron School emphasizes collaboration, innovation, and a commitment to student success. A strong alignment with the company's mission and values is crucial.
PracHub interview research ↗What is the typical timeline from initial screen to offer?
The timeline can vary, but candidates can generally expect feedback within a few weeks after the final interview. The process may include multiple rounds, so patience and preparedness are key.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
Yale University Data Scientist Interview Questions & Guide 2026
Timed practice for Yale University data scientist candidates who freeze under a clock: short SQL and probability sets run end to end, then reviewed against a written rubric.
Universitätsklinikum Tübingen Data Scientist Interview Questions & Guide 2026
A prep plan for Universitätsklinikum Tübingen data scientist candidates built on spaced review: write a full answer before opening any solution, keep a running list of the assumptions you failed to state, and repeat every missed exercise th
University of Michigan Data Scientist Interview Questions & Guide 2026
Metric-diagnosis practice for University of Michigan data scientist candidates: a number moves week over week and you work out why, splitting by segment, ruling out a logging or release change, and separating a mix shift from a real change
University of Florida Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for University of Florida data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.