At Paylocity, a Data Scientist is a core driver of innovation within the Product & Technology division. Paylocity is an industry-leading provider of cloud-based Human Capital Management (HCM) and payroll software. The data science team is responsible for turning vast amounts of workforce, financial, and behavioral data into high-impact AI and machine learning solutions. These solutions deliver automation, predictive insights, and intelligent assistance to millions of users daily, helping organizations make smarter human capital decisions.
As a member of this team, you will work on highly complex, diverse data science problems. The team's portfolio spans natural language processing (NLP), supervised learning, time-series forecasting, anomaly detection, and generative AI. Whether building models that predict employee turnover, automating complex payroll workflows, or developing agentic AI applications powered by Large Language Models (LLMs), your work directly influences the product features that keep Paylocity at the forefront of the HCM market.
This role requires a unique blend of deep technical expertise and strong business acumen. You will not only write production-grade machine learning code in Python and design scalable pipelines on cloud platforms like AWS and Azure, but you will also collaborate closely with Product Managers, Software Engineers, and Enterprise Architects. Successful candidates are those who can navigate ambiguous business challenges, translate them into structured data science problems, and deliver robust, scalable prototypes.
Recruiter Screening
reportedA screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.
What to demonstrate
- Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
- Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
- Whether timeline, location and compensation expectations make the rest of the loop worth scheduling
How to prepare
- Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
- Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
- Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice
Technical Screen
reportedA handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.
What to demonstrate
- Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
- Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
- Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.
How to prepare
- Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
- Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
- If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows
Take-Home Case Study
reportedA take-home is graded as an argument, not as a notebook. Somebody reads the submission without you in the room, so every choice has to survive on the page: why the question was framed this way, and what was deliberately left out. The gap between a strong and a weak submission is almost never model quality. It is whether the writeup names the specific question it answers and commits to a recommendation, including what evidence would overturn it. A high-accuracy model attached to no conclusion reads as effort that stopped before the decision.
What to demonstrate
- Whether the question you answered is stated outright, and whether it is the question the prompt posed rather than an easier neighbour of it
- Whether the recommendation is specific enough to act on, with the uncertainty attached to it instead of parked in a caveats section at the end
- Whether analytical choices such as the metric definition, the population filter and the time window are justified in the prose, not merely visible in code
How to prepare
- Take a dataset you have already worked with, write the one-paragraph conclusion first, then check whether the analysis you were planning actually supports it and cut whatever does not
- Practise stating a metric in one sentence that fixes the population, the time window and the denominator, then confirm your query computes exactly that sentence and nothing adjacent to it
- Hand a draft to someone outside the problem and ask them to tell you back what you recommended and why; anything they cannot recover is not on the page yet
Panel Interview
reportedWhere a loop ends with a senior leader, that conversation is rarely another skills test. The technical signal already exists by then, so the questions tend to open up: what you would look at first, where a metric you have heard about could mislead, what you would push back on. The decision being made is scope, which in practice means level and how much you would be trusted to own unsupervised. Treating it as a formality is the usual mistake. An open question late in the day is still being scored, and a vague answer reads as someone who has not run anything themselves.
What to demonstrate
- Whether your view of the business has anything specific behind it, given that you are working only from what is public and are expected to say so
- Whether the scope of work you describe owning matches the scope of the role, instead of sitting a level below it
- Whether you can disagree with something concrete and stay useful about it, rather than agreeing with everything said in the room
- Whether your questions are ones only this person could answer, as opposed to ones the recruiter already covered
How to prepare
- Build one view you could defend for two minutes using only public information: what the funnel probably looks like, which metric likely drives decisions, and where that metric could mislead. Being wrong for a stated reason survives this round; having no view does not
- Write down the largest piece of work you have owned from question to decision, who else touched it, and what you decided alone, then check that it reads at the level you are interviewing for
- Prepare one thing you would want changed if you joined and phrase it as a question rather than a verdict, so it opens a conversation instead of closing one
PracHub editorial advice for the preparation topics above.
Randomising an experiment at the user level when users share an account
Two problems fire at once. Colleagues in one workspace see each other's work and talk to each other, so a treated user changes the behaviour of a control user in the same account, which violates the no-interference assumption and biases the estimate toward zero. Separately, outcomes within an account are strongly correlated, so the effective sample size is roughly n / (1 + (m - 1) * rho) for m users per account and intra-class correlation rho, not n. With rho around 0.3 and twenty users per account that is a design effect near 6.7, meaning a user-level confidence interval is about two and a half times narrower than it should be and results cross significance thresholds on noise alone. Randomise the account and cluster the standard errors.
Reporting a mean over accounts when account revenue is heavy-tailed
When a small number of accounts hold most of the revenue, the sample mean is dominated by whichever of them happens to be in the sample, and the sample variance keeps growing as more data arrives instead of stabilising. In that regime the usual central-limit-based confidence interval understates uncertainty, and a single renewal or a single large account's batch job can flip the sign of a measured effect. The fixes are to pre-register a winsorisation or capping rule before looking at the outcome, to report account counts crossing a threshold alongside the revenue figure, or to define the estimand on a bounded transform. Choosing the cap after seeing the result is a separate and worse problem, because the cap then encodes the answer.
Over-explaining the method and under-explaining the implication
Lead with the answer and what you would do about it, then give the approach when asked. Roughly one sentence of method per three of implication is the right ratio for a stakeholder-facing answer; the interviewer already knows what a regression is.
Sizing estimates built on unnamed, unrevisable assumptions
Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.
Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.
Bootstrap a confidence interval for net revenue retention
You have one row per account with arr_start_cents (ARR twelve months ago) and arr_end_cents (ARR today, zero if churned), covering the fixed cohort of accounts that had ARR twelve months ago. Net revenue retention is sum(arr_end_cents) / sum(arr_start_cents). Write a nonparametric bootstrap from scratch, without scipy.stats.bootstrap: resample accounts with replacement, recompute the ratio of sums on each resample, and return the point estimate with a 95% percentile interval from 10,000 resamples. Also report the interval you would get from the mean of per-account ratios, and explain the difference.
Approach
- Resample the account, because the account is the unit the estimand is defined over. One bootstrap draw is a vector of account indices and both numerator and denominator are recomputed from that same draw; resampling the two sides independently destroys the within-account correlation that makes a ratio estimator stable.
- Vectorise the draws: idx = rng.integers(0, n, size=(B, n)), then end[idx].sum(axis=1) / start[idx].sum(axis=1). A 10,000 by n index matrix is usually far cheaper than a Python loop; if the matrix is too large for memory, chunk over B rather than reverting to a loop.
- Take the interval from np.quantile(ratios, [0.025, 0.975]). The percentile interval differs from estimate +/- 1.96 * bootstrap SE whenever the resample distribution is skewed, which it will be here, and the skew is the thing you want represented.
- Compute the mean-of-ratios version on the same resamples, and state the exact relationship rather than guessing which of the two is larger. With r_i = arr_end_i / arr_start_i, the ratio of sums is the arr_start-weighted mean of exactly those r_i, so sum(end)/sum(start) - mean(r) = Cov(arr_start, r) / mean(arr_start) using the population covariance. The gap is positive when larger accounts retain and expand better than smaller ones, and negative when they do not; a cohort whose small accounts churn at a higher rate has positive covariance, which puts the mean of per-account ratios BELOW the ratio of sums. Requires arr_start_i > 0 for every account, which the fixed-cohort definition guarantees; r_i is floored at 0 and unbounded above, so a handful of 4x expansions among small accounts can flip the sign. Compute the covariance and report it instead of asserting a direction.
- Report the interval width beside the concentration of the cohort. If the largest account is 12% of starting ARR, a narrow interval is evidence that the resampling unit is wrong rather than evidence that the estimate is precise.
Follow-up
- The cohort has 800 accounts and the largest is 12% of starting ARR. How much do you trust a percentile interval here?
- How would you extend this to an interval on the year-over-year change in NRR?
- Two accounts merged mid-window and one contract was co-termed into the other. How do you keep the cohort fixed?
Join daily usage to the contract version live that day
fct_usage_daily has account_id, usage_date and net_amount_cents. fct_subscription_period has account_id, subscription_period_id, plan_tier, term_start_date, term_end_date, arr_cents and is_current, with one row per contract term version and every amendment inserting a new row. Attach to each usage row the subscription_period_id whose term brackets usage_date (term_start_date <= usage_date <= term_end_date). pd.merge_asof and an interval-condition merge are unavailable; use sorting and numpy.searchsorted. Then report monthly net revenue by plan_tier. Terms for one account do not overlap, and usage may fall outside every term.
Approach
- Say out loud why the cheap version is wrong: joining on is_current stamps today's plan tier onto last year's usage, so every account that upgraded has its history reclassified and revenue-by-tier becomes a function of when the query ran.
- Sort each account's terms by term_start_date and use np.searchsorted(term_start, usage_date, side='right') - 1 to get the last term that started on or before the usage date. Do it per account group, or globally after encoding (account_id, date) into one monotone key.
- searchsorted only enforces the left edge. Validate the right edge afterwards — usage_date <= the candidate's term_end_date — and set the match to NA where it fails. That NA is usage in a gap between contracts and must stay visible instead of being folded into the expired term.
- Assert non-overlap before trusting the lookup, and write the assertion so it is capable of passing. prev_end = terms.groupby('account_id').term_end_date.shift() is NaT on each account's first row, and NaT < Timestamp evaluates to False rather than NA, so a comparison followed by .fillna(True) has nothing left to fill and the assertion fires on every account's first term whatever the data looks like. Guard the null yourself: assert (prev_end.isna() | (prev_end < terms.term_start_date)).all(). The failure mode of getting this wrong is not a false alarm you notice once — it is an assertion someone deletes because it never passes, after which overlapping terms make searchsorted return one of them with no trace in the output.
- Aggregate after the join, grouping by (usage_date month, plan_tier) with dropna=False so the unmatched bucket appears as its own row and the total still ties to the ungrouped sum of net_amount_cents.
Worked solution 35 min
- terms = terms.sort_values(['account_id','term_start_date']); prev_end = terms.groupby('account_id').term_end_date.shift(); assert (prev_end.isna() | (prev_end < terms.term_start_date)).all()
- Per account group: idx = np.searchsorted(g.term_start_date.values, u.usage_date.values, side='right') - 1; rows with idx < 0 are unmatched.
- Gather subscription_period_id, plan_tier and term_end_date by positional index, then null the match wherever usage_date > the gathered term_end_date.
- monthly = joined.assign(month=joined.usage_date.dt.to_period('M')).groupby(['month','plan_tier'], dropna=False).net_amount_cents.sum()
Follow-up
- An amendment takes effect on the 17th of a month. How do you report that month's revenue by tier?
- What changes if terms can overlap because of a co-term amendment?
- How would you verify this against a SQL implementation using a BETWEEN condition?
Implement seven-day activation rate from its written definition
Given dim_account (account_id, created_at, is_internal, is_current) and fct_api_request (account_id, request_at, http_status, api_key_id, traffic_class), implement activation_rate(accounts, requests, week_start). Definition: the numerator is accounts whose first request with http_status < 400, api_key_id not null and traffic_class != 'synthetic_monitor' occurs no later than 168 hours after created_at; the denominator is non-internal accounts created during the ISO week starting week_start. All timestamps are tz-aware UTC. Return the rate and both counts, and refuse to report a week until every account in it has had its full 168 hours.
Approach
- Reduce dim_account to one row per account_id before joining anything. It is a type 2 dimension, so several versions of the same account exist; joining the versioned table to requests multiplies the denominator by the number of plan changes an account happened to make.
- Build the denominator first and freeze it: is_internal == False and week_start <= created_at < week_start + 7 days. Everything after this is a filter on the numerator only, because an account that never sent a request must still sit in the bottom of the fraction.
- Filter requests to qualifying rows and only then take groupby('account_id').request_at.min(). The first qualifying request is not the same object as the global first request filtered afterwards, and the two answers differ for every account whose first call was a 4xx.
- Left-join the first qualifying timestamp onto the cohort and test (first_ok - created_at) <= Timedelta(hours=168). NaT propagates to False in that comparison, which is the behaviour you want, but assert it rather than assume it.
- Guard reportability explicitly: if week_start + 7 days + 168 hours exceeds the maximum request_at in the data, the week is censored and will read as a drop, so return None or raise rather than emit a number.
Follow-up
- Median time-to-first-call is more informative. What breaks if you take the median over activated accounts only, and what estimator fixes it?
- How would you decide whether 168 hours is the right window rather than 72 or 336?
- An account signs up, does nothing for 20 days, then integrates heavily. Where does it land in this metric, and is that what you want?
Sessionise interactive API traffic with a thirty-minute inactivity gap
fct_api_request carries account_id, user_id (null for service accounts), request_at, traffic_class and http_status. Using only traffic_class = 'interactive' rows with a non-null user_id, group each seat's requests into sessions on a 30-minute inactivity threshold: a request more than 30 minutes after the previous request from the same (account_id, user_id) opens a new session. For one ISO week return, per account, the session count, the median session duration in minutes and the median requests per session. A single-request session has a duration of zero.
Approach
- Filter first: traffic_class = 'interactive' and user_id IS NOT NULL. Machine traffic has no sessions in any useful sense, and leaving CI or batch rows in produces sessions that are really cron schedules.
- Get the previous timestamp with LAG(request_at) OVER (PARTITION BY account_id, user_id ORDER BY request_at). Partitioning by user_id alone stitches one person's work across two different accounts into one fabricated session, because a human holds memberships in several accounts.
- Flag a boundary where the lag is NULL or request_at - lag > interval '30 minutes'. Decide and state whether exactly 30 minutes continues the session; strictly greater is the conventional choice and needs to be written down either way.
- Assign session ids with SUM(boundary::int) OVER (PARTITION BY account_id, user_id ORDER BY request_at ROWS UNBOUNDED PRECEDING), the standard running-count construction for islands.
- Roll up to sessions with min(request_at), max(request_at) and count(*), then to accounts with percentile_cont(0.5) WITHIN GROUP (ORDER BY ...). Use medians, not means: session length is strongly right-skewed and one long-running client dominates the average.
Worked solution 30 min
- Build the filtered week: interactive rows with user_id IS NOT NULL inside [week_start, week_start + 7 days).
- Add prev_at via LAG(request_at) OVER (PARTITION BY account_id, user_id ORDER BY request_at) and derive is_new_session = (prev_at IS NULL OR request_at - prev_at > interval '30 minutes').
- Add session_seq = SUM(is_new_session::int) OVER (PARTITION BY account_id, user_id ORDER BY request_at ROWS UNBOUNDED PRECEDING).
- Aggregate to sessions by (account_id, user_id, session_seq) taking min, max and count, with duration_minutes = EXTRACT(EPOCH FROM max - min) / 60.
- Aggregate to accounts with count(*) AS sessions, percentile_cont(0.5) WITHIN GROUP (ORDER BY duration_minutes) and percentile_cont(0.5) WITHIN GROUP (ORDER BY request_count).
Follow-up
- Sessions belonging to colleagues in one account are correlated. What does that do to a t-test on session length across an experiment arm?
- A long-poll or streaming endpoint keeps a connection open for hours. How do you stop it reading as one twelve-hour session?
Find paid seats that never called the API
dim_user_membership carries user_id, account_id, seat_type, is_service_account, deactivated_at and last_seen_at. fct_api_request carries user_id, which is null whenever the caller is a service account or an unattended key, plus account_id, request_at and http_status. For a single account, list every licensed_paid seat with is_service_account = false and deactivated_at null that issued no request at all in the trailing 90 days, returning user_id and last_seen_at. A first draft writes NOT IN against a subquery over fct_api_request.user_id. State exactly what that draft returns and why, then write the correct query.
Approach
- Read the column comment first: user_id is nullable on the fact, so the subquery almost certainly contains at least one NULL for any account that runs a service account or an unattended key.
- Work the three-valued logic out loud. x NOT IN (a, NULL) expands to NOT (x = a OR x = NULL); the second disjunct is UNKNOWN, so for any x not equal to a the whole predicate is UNKNOWN and the row is filtered out. The draft returns zero rows, which reads as full seat utilisation.
- Rewrite as NOT EXISTS with the 90-day and status predicates inside the correlated subquery. Putting them in the outer WHERE instead turns the anti-join into a different question and silently changes the answer.
- Correlate on both user_id and account_id. A membership is (user_id, account_id) and one person can hold memberships in several accounts, so correlating on user_id alone marks a seat as active because that human was busy somewhere else.
- Filter the seat side to seat_type = 'licensed_paid', is_service_account = false and deactivated_at IS NULL, and say what the resulting count means next to contracted_seats on the current subscription row.
Follow-up
- Adding AND user_id IS NOT NULL to the subquery also fixes NOT IN. Why is NOT EXISTS still the form you would leave in the repository?
- last_seen_at looks like a shortcut for the whole query. What does it actually record, and where does it disagree with API activity?
- This is a seat-reduction risk list. What threshold would you attach before handing it to an account team, and what happens to seats below it?
Triage a sample ratio mismatch before reading the result
An account-randomised test assigned arms 50/50 by hashing account_id. Exposure logging recorded 4,200 accounts: 1,987 in treatment and 2,213 in control. A chi-square goodness-of-fit test against the intended 2,100/2,100 split gives chi-square(1) = 12.2, p about 0.0005. The analyst reports the primary metric up 6% in treatment and asks to ship. Explain what the imbalance implies about that 6%, rank the causes you would investigate first given this domain's data, and name for each the single query against fct_api_request or dim_account that confirms or eliminates it.
Approach
- State the rule before diagnosing anything: a sample ratio mismatch below a 0.001 alarm threshold invalidates the readout. Whatever mechanism removed 113 accounts from one arm almost certainly removed them non-randomly, which makes it a selection effect on the outcome. The 6% is not to be adjusted, caveated or shipped; it is unusable until the mechanism is named.
- Verify the test is testing the right thing. chi-square = (1987 - 2100)^2 / 2100 + (2213 - 2100)^2 / 2100 = 6.08 + 6.08 = 12.16 on one degree of freedom. Confirm the denominator is the intended assignment count rather than the observed total, and that you are testing assignment rather than an analysis population that has already been filtered.
- Rank causes by how this domain actually breaks rather than by textbook order: is_internal accounts filtered after assignment instead of before; exposure logged on a code path that returns 5xx more often in one arm, silently dropping those accounts; assignment recorded at first request but exposure at a later event, so accounts that churned in between appear in only one arm; dim_account type-2 versioning giving one account_id two surrogate keys and two hash inputs; deployment_model = 'self_hosted' accounts that never emit exposure telemetry at all.
- Attach one discriminating query to each: count arms before the is_internal filter; compare the http_status >= 500 rate by arm on the exposure endpoint in fct_api_request; compare the distribution of created_at and churned_at by arm in dim_account; count distinct account_sk per account_id inside the experiment window; cross-tabulate arm against deployment_model.
- Compare counts at three fixed stages, assignment, first exposure, and analysis population, and localise the divergence to one of them. The stage where the arms first separate names the subsystem, and everything downstream of it is a symptom.
- Report the outcome as abort, fix, rerun, and be explicit about what survives: the variance estimate for re-powering, the instrumentation fix, and nothing whatsoever about the effect size.
Worked solution 20 min
- Recompute the statistic: two terms of 113^2 / 2100 = 6.08, total 12.16, p about 0.0005 on one degree of freedom, below the 0.001 alarm line.
- Pull arm counts at assignment, at first exposure, and in the analysis population, and find the first stage at which they diverge.
- At that stage, cross-tabulate arm against is_internal, deployment_model, and the 5xx rate on the exposure endpoint.
- Write the abort note naming the mechanism, the corrected code path, and the rerun date, with the effect estimate explicitly withheld.
Follow-up
- The imbalance disappears once you restrict to accounts with at least one successful request. Does that fix the experiment or confirm the bug?
- What alarm threshold would you set for this check, and why is 0.05 the wrong one for a diagnostic you run on every experiment every day?
- How would you detect a mismatch confined to one segment while the overall split looks clean?
Measure switching cost when switching cost is not observable
Leadership wants switching cost tracked as a leading indicator of renewal. Nothing in the warehouse records it. Available: fct_api_request (account_id, workspace_id, environment, api_key_id, sdk_name, sdk_version, endpoint, traffic_class, http_status, request_at) and dim_account (account_id, employee_band, deployment_model, is_internal). Propose a proxy, state the direction and likely size of its bias, and name one decision the proxy is good enough for and one it is not. Deliver the proxy definition, the written bias statement, and the validation you would run against observed renewal outcomes.
Approach
- Say first that switching cost is unobservable in this data and that the deliverable is a biased correlate with its bias written down, not a measurement. Anything presented as a direct measure of switching cost is already wrong before the SQL starts.
- Define the proxy as production integration breadth per account over a trailing 28 days: distinct endpoint route templates, distinct api_key_id, and distinct workspace_id with environment = 'production', all restricted to http_status < 400 and traffic_class in ('interactive','batch'). Breadth, not volume, because request volume is one CI configuration change away from an order of magnitude.
- State the biases with their mechanisms and their sign. Upward with account size, because breadth correlates with employee_band, so the proxy ranks large accounts as sticky whether or not they depend on anything. Downward for deployment_model = 'self_hosted', whose traffic does not all cross the managed gateway, so their breadth is systematically understated. Blind to criticality: one endpoint carrying a production billing path is a larger switching cost than twenty endpoints behind a read-only dashboard, and nothing in this data distinguishes them.
- Handle the bias by stratifying rather than by pretending it is gone. Report the proxy within employee_band and deployment_model strata and state explicitly that cross-stratum comparisons are not supported by the construction.
- Validate against the only outcome that is actually observed: renewal on the renewal-eligible base. Within strata, report renewal rate by proxy quintile for accounts whose term_end_date has passed plus a 45-day grace, and report discrimination at the operating point a capacity-bound team can work rather than a global AUC.
Follow-up
- An account's endpoint breadth drops 40% in a week. Name three explanations that have nothing to do with reduced dependency.
- Which decision would you refuse to make on this proxy, and what evidence would you need before making it?
Error rate halves while severe support tickets double
The fleet-wide customer-visible error rate fell from 1.8 percent to 0.9 percent, while sev1 and sev2 tickets and reopened_count rose across the same fortnight. Using fct_api_request (account_id, environment, sdk_name, traffic_class, http_status, request_at, billable_units), fct_usage_daily (account_id, sku_code, usage_date, billable_quantity) and fct_support_ticket (account_id, severity, opened_at, reopened_count, linked_incident_id), determine whether reliability improved, and if not, identify precisely which rows are missing and from when. Deliverable: a diagnosis backed by an independent corroborating source.
Approach
- Distrust an improvement that contradicts an independent operational signal. Two sources disagreeing is itself the finding; decide which one is more likely to be broken before explaining either.
- Recompute the rate as the metric tree defines it, per account first and then as the share of accounts above the reliability target. A single global average is dominated by whichever account sends the most traffic, so a fleet number can fall while a quarter of accounts get worse.
- Audit for missingness rather than for badness: count fct_api_request rows per hour split by environment, sdk_name and status class, indexed against the trailing same-hour baseline. A partial ingestion failure shows as a step drop confined to one slice, not a uniform decline.
- Reconcile against a source the request pipeline does not feed, such as implied request volume from fct_usage_daily for the same accounts and dates. If the usage table is flat while request rows fell, rows are missing rather than traffic.
- Test whether the missingness is differential by status, which is the mechanism that fakes an improvement: if 5xx rows are written on a path that stopped while 2xx rows were unaffected, the numerator falls faster than the denominator and the rate drops with nothing improving.
- Close with the affected window, the affected slice and a restated series marked unreliable across that window, rather than a silently patched number.
Follow-up
- The missing rows are unrecoverable. How do you present that fortnight in a series people compare week over week?
- What monitor would have caught this within an hour, and what is its false-positive cost on a normal quiet weekend?
Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.
Prepare, practise & reflect
One practical outcome each day. Spend longer where you need it.
0 / 7 done01Breadth pass: query fluency
- Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
- For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
- Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.
Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.
Practice prompt ↗Practice prompt ↗Worked solution ↗02Breadth pass: statistics and inference
- Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
- Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
- Rewrite the two weakest answers the following morning from memory in full sentences.
Deliverable: Ten graded answers with an honest count of exact hits.
Practice prompt ↗Practice prompt ↗03Breadth pass: modelling
- Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
- Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
- Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.
Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.
Practice prompt ↗Practice prompt ↗04Breadth pass: product judgement
- Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
- For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
- Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.
Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.
Practice prompt ↗Practice prompt ↗Worked solution ↗05Depth, first area
- Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
- Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
- Re-solve the two you failed the same evening with notes closed.
Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.
Practice prompt ↗06Depth, second area, and the seam between them
- Repeat the depth protocol on the second-ranked area with the same six-problem structure.
- Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
- Solve your own combined problem end to end and note where the handoff between the two areas cost you time.
Deliverable: One combined problem, solved end to end, with the handoff failure written down.
Practice prompt ↗07Integration and re-measurement
- Re-run the six prompts from day one under the same clock and compare both correctness and time.
- Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
- Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.
Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.
Practice prompt ↗Worked solution ↗Expand any day for tasks and deliverables. Your progress is saved on this device.
Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.
Walk through an analysis you later discovered was wrong
Six weeks ago you reported that consumption fell 9 percent in the last week of the month, and a team spent a sprint investigating the cause. The fall was an artefact: rows in fct_usage_daily land late and are restated in place, and you queried before the tail had settled. Describe how you found the error, what you told the people who acted on it, and the control you put in place so this class of mistake cannot reach a dashboard again. Be specific about how the settling window was measured.
Approach
- The interviewer is probing whether you self-report errors before someone else finds them, and whether your fix is structural rather than a promise to be more careful. Say plainly that the number was wrong and that a sprint was spent on it, before describing any diagnosis.
- Establish the artefact quantitatively instead of asserting that data lands late. For each usage_date, compare the total as of first_written_at against the settled total and read the settling time off that curve, for example 97 percent of final by day three and 99.5 percent by day five.
- Correct the record the same day, in the channel the original number went out in, to the same audience. The cost of the wasted sprint belongs in the correction, not in a footnote.
- Make the fix structural: exclude a trailing lag window from every reportable figure, and make the reporting view return no rows inside that window rather than returning partial ones. A dashboard that shades unsettled days still gets read as a decline.
- State what generalises. Any fact table restated in place has this failure mode, so the guard belongs at the source rather than on the one dashboard that embarrassed you. A strong answer ends with the class of error closed; a generic one ends with a lesson learned.
Follow-up
- How did you choose the completeness threshold behind the lag window, and what would make you recalibrate it?
- What did you say to the team that lost the sprint, and what did they say back?
- Is there a legitimate case for showing the unsettled tail at all, and to whom?
Announce a metric fix that cuts the headline number
Weekly active organisations, the count on the company dashboard, has never excluded rows where dim_account.is_internal is true, and it counts traffic with traffic_class in synthetic_monitor and load_test. Correcting both reduces that count by 11 percent and removes most of the growth reported over two quarters. The figure appears in a board deck and in two teams' quarterly goals, one written on the count and one on the weekly active organisation ratio, whose denominator is accounts whose account_status was in ('trial','free','active_paid') through the week. Decide the order in which you tell people, what the dashboard shows during the transition, and what you propose happens to goals already set against the old definition.
Approach
- The interviewer is probing whether you can land a correction as an operational change with a plan attached, rather than as an announcement other people then have to clean up after.
- Quantify each exclusion separately before telling anyone: internal accounts, synthetic monitors, load tests. Three known quantities are a discussion; one alarming total is an argument.
- Be precise about which side of the metric each exclusion touches, because one team's goal is on a count and the other's is on a ratio. The traffic-class filters remove requests, so they shrink the numerator only. Dropping internal accounts removes them from the ratio's denominator as well, since internal accounts carry ordinary account_status values and therefore sit in that denominator. Internal accounts are active in almost every week while the real base is not, so the numerator loses a larger share than the denominator and the ratio falls by less than the count does. Compute both and say which one the 11 percent is before anybody assumes.
- Check whether the trend changes, not only the level. A constant 11 percent shift is a rebasing and nothing more. A shift that widens over time means the reported growth was partly internal or synthetic, which makes the existing goals unachievable as written and changes what you are asking teams to do.
- Sequence the disclosure: the metric owner and the two teams whose goals move first and privately, then the board channel with a written bridge, then the dashboard. The dashboard is last because a number that changes without explanation is read as instability rather than as a fix.
- Run both series for one reporting period with the bridge visible, restate history rather than letting the series break at a date, and set the date the old series is removed.
- Propose the goal treatment yourself: rebase each target by the shift measured on the metric that target is written against, rather than leaving each team to negotiate individually, which is where corrections of this kind usually die.
Follow-up
- One team's quarterly goal is now unreachable. Rebase the target or let it miss, and what does each choice teach the organisation?
- How would this have been caught when the metric was first defined?
- What else on that dashboard shares this failure mode, and how would you find out this week?
Allocate one analyst week across three competing requests
Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.
Approach
- The interviewer is probing whether you prioritise on decision timing and reversibility or on who asked most forcefully. Sort by the date each decision is actually taken and by what the default outcome is if nothing arrives.
- Apply that sort concretely. The Thursday readout has a hard irreversible deadline and no value afterwards. The pricing review has three weeks of slack. The renewal list has a rolling deadline set by term_end_date, so part of it is urgent this week and the rest is not, which means it can be split rather than deferred whole.
- Find the cheapest sufficient version of each request rather than the full version. The readout goes in full. The renewal list ships as a filtered query over renewal-eligible accounts ranked by two inspectable signals rather than as a model. The margin work is scoped to the accounts that dominate the pricing decision, since revenue is heavily skewed and the tail will not change the conclusion.
- Make the trade visible in one written note to all three at once, with dates. Telling each person separately that they are the priority is how an allocation becomes a credibility problem.
- Refuse something explicitly and say why. The model version of the renewal list is the usual candidate, because it cannot be evaluated without a holdout nobody has agreed to yet, and building it this week forecloses that.
- Leave slack. A plan with none is a plan to miss the one deadline that cannot move.
Follow-up
- The sales leader escalates to your manager. What did you already do that makes that a short conversation?
- Which of the three deadlines would you push back on, and what exactly would you ask for?
- What would you change about how these requests reach you so next week is not the same?
- 01
Six weeks ago you reported that consumption fell 9 percent in the last week of the month, and a team spent a sprint investigating the cause. The fall was an artefact: rows in fct_usage_daily land late and are restated in place, and you queried before the tail had settled. Describe how you found the error, what you told the people who acted on it, and the control you put in place so this class of mistake cannot reach a dashboard again. Be specific about how the settling window was measured.
- 02
Weekly active organisations, the count on the company dashboard, has never excluded rows where dim_account.is_internal is true, and it counts traffic with traffic_class in synthetic_monitor and load_test. Correcting both reduces that count by 11 percent and removes most of the growth reported over two quarters. The figure appears in a board deck and in two teams' quarterly goals, one written on the count and one on the weekly active organisation ratio, whose denominator is accounts whose account_status was in ('trial','free','active_paid') through the week. Decide the order in which you tell people, what the dashboard shows during the transition, and what you propose happens to goals already set against the old definition.
- 03
Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.
Is this an official Paylocity interview guide?
No. It is PracHub's own research and practice material for the Data Scientist role at Paylocity. Rounds and questions reflect what candidates have reported, not a process Paylocity has published, and they change over time. Confirm the current format and scope with your recruiter.
PracHub interview research ↗How difficult is the Paylocity Data Scientist interview process?
Candidates generally rate the process as average to difficult. The individual interview rounds are straightforward and practical, but the 8-hour timed take-home case study requires significant focus, time management, and technical execution to pass.
PracHub interview research ↗Can I use my own local development environment for the take-home challenge?
Yes. You can use your preferred IDE and libraries, but you must ensure your code is reproducible, clean, and well-documented. You will be expected to walk the technical panel through your repository and explain your code structure line-by-line.
PracHub interview research ↗What is the remote work policy for Data Scientists at Paylocity?
Most data science roles at Paylocity are fully remote within the United States. However, you must be available to work during designated business hours and collaborate effectively using virtual communication tools.
PracHub interview research ↗Why does Paylocity place so much emphasis on the take-home case study?
Paylocity values practical, hands-on capability over theoretical memorization. The take-home challenge allows the team to evaluate your real-world coding style, engineering practices, structured problem-solving, and communication skills in a simulated project environment.
PracHub interview research ↗Sources & methodology 3 sources ↗
No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.
- 01PracHub interview research ↗
PracHub editorial research into this company and role, maintained with this guide. Candidate-reported, not an employer publication.
PracHub page · Accessed 2026-09-22 - 02PracHub Data Scientist practice ↗
Cross-company practice questions for this role.
PracHub page · Accessed 2026-09-22 - 03PracHub interview preparation framework ↗
The framework the preparation plan follows.
PracHub page · Accessed 2026-09-22
Explore related engineering environments and compare the preparation focus.
[24]7.ai Data Scientist Interview Questions & Guide 2026
Modeling drills for [24]7.ai data scientist prep: choosing a baseline, reading regression coefficients honestly, handling class imbalance, and defending a model you would actually ship.
Xcellent Technology Solutions Data Scientist Interview Questions & Guide 2026
Modeling drills for Xcellent Technology Solutions data scientist prep: choosing a baseline, reading regression coefficients honestly, handling class imbalance, and defending a model you would actually ship.
Wolters Kluwer Data Scientist Interview Questions & Guide 2026
Causal-inference exercises for Wolters Kluwer data scientist prep: difference-in-differences, matching, instrumental variables, and knowing when observational data simply cannot answer the question asked.
Zscaler Data Scientist Interview Questions & Guide 2026
Data-quality drills for Zscaler data scientist prep: survivorship and selection bias, leakage through time-ordered features, and extreme values where provenance decides the call. A logging fault or unit slip gets corrected or dropped; a gen