Delayed Outcomes, Censoring, And Survival Analysis

What's being tested

The interviewer checks your ability to reason about time-to-event outcomes, measurement bias from delayed outcomes and censoring, and to choose valid estimators and pre-analysis plans for experiments or cohort analyses. Meta cares because many product metrics (conversions, retention, lifetime value) arrive slowly; a naive analysis produces biased lift estimates and bad decisions. Expect to justify estimands, show survival-aware estimators, and communicate assumptions and sensitivity checks.

Core knowledge

Right-, left-, and interval-censoring: right-censoring occurs when the event hasn't happened by observation end; interval-censoring when only known to occur between two checks; left-censoring when the event happened before observation start. Assumptions differ.
Non-informative censoring: most survival methods require censoring independent of event time conditional on covariates; violation induces bias — plan sensitivity checks and IPCW adjustments.
Survival function and hazard: $S(t)=P(T>t)$ and hazard $h(t)=\lim_{\delta\to0}P(t\le T<t+\delta\mid T\ge t)/\delta$ . Causal interpretation differs from cumulative incidence.
Kaplan-Meier estimator: nonparametric product-limit estimator for $S(t)$ under right-censoring; use for plotting survival curves and computing survival probabilities at fixed times.
CoxPH model: semi-parametric hazard model $\lambda(t|X)=\lambda_0(t)\exp(X\beta)$ ; returns hazard ratios but relies on proportional hazards (PH); check PH via Schoenfeld residuals.
Restricted mean survival time (RMST): $\text{RMST}(\tau)=\int_0^\tau S(t)\,dt$ is an interpretable alternative to hazard ratios for a prespecified $\tau$ (good when PH fails).
Inverse-probability-of-censoring weighting (IPCW): weight each observed outcome by $1/\hat{G}(t)$ where $\hat{G}$ is censoring survival; useful to estimate risk at fixed horizon under informative follow-up design.
Parametric models (Weibull/Exponential/Log-normal, AFT): useful when proportional hazards fail or for extrapolation; AFT models give multiplicative effects on median time-to-event.
Competing risks: if multiple mutually exclusive event types exist, use cumulative incidence functions, not naive KM, to avoid overestimating event probabilities.
Experiment design tactics: pre-specify estimand (e.g., intent-to-treat 30-day risk), analysis window τ, censoring rule, and primary estimator (KM+RMST or IPCW); power/sample-size routines must account for censoring via expected event rates over follow-up.
Bias-variance tradeoff with window τ: longer τ reduces censoring bias but increases time-to-decision and variance; report sensitivity across several τ and RMST differences.
Discrete-time survival / grouped-time: when timestamps are coarse or checks periodic, use complementary log-log or logistic models on person-period data; this handles interval-censoring practically.
Bootstrap and robust SEs: heavy censoring or small samples require bootstrap CIs for RMST or IPCW estimates; robust sandwich SEs for Cox when model mis-specification is possible.

Worked example — measuring treatment effect on 30‑day conversion

Frame: ask clarifying questions — define the event (first conversion), treatment assignment timing, and whether follow-up ends at 30 days for all users; declare the estimand: intent‑to‑treat difference in 30‑day conversion probability or RMST up to $\tau=30d$ .

Skeleton answer pillars: (1) Data setup: build survival table (time-to-event or time-to-censoring). (2) Primary estimator: use Kaplan-Meier to estimate $S(t)$ in each arm and report risk difference at 30 days (1− $S(30)$ ). Optionally report RMST(30) difference. (3) Adjustments: apply IPCW or Cox with baseline covariates to improve precision; check PH. (4) Sensitivity: show estimates for τ=14,30,60 days and perform an analysis under worst-case informative censoring.

Tradeoff to flag: choosing RMST vs hazard ratio — RMST is more interpretable and robust when PH fails, but requires selecting τ (domain-driven). If I had more time, I'd pre-register the estimand, perform subgroup heterogeneity checks, and simulate power under plausible censoring distributions.

A second angle — interval‑censored engagement time with daily aggregation

Different constraints: timestamps are only daily, and the event is known to happen during a day (interval-censoring). Nonparametric Kaplan-Meier is invalid for interval-censoring; instead (1) use Turnbull estimator or (2) convert into discrete-time person-period dataset and fit complementary log-log or discrete-time logistic models for hazard. For experiments, estimate discrete cumulative incidence at a prespecified day and use IPCW adapted to interval-censoring. Emphasize that assumption changes (interval widths matter) and that coarse intervals increase uncertainty — you should communicate this and, if possible, refine instrumentation to get finer timestamps.

Common pitfalls

Pitfall: Treating censored units as non-events. Counting censored users as failures underestimates event probability and biases treatment effects toward zero; always use survival-aware estimators or justify why censoring is irrelevant.

Pitfall: Interpreting hazard ratios as risk ratios. A hazard ratio does not equal a relative risk at a fixed time unless hazards are proportional and rare; prefer RMST or absolute risk differences for product decisions.

Pitfall: Failing to pre-specify estimand and τ. Changing the analysis window after seeing data (lengthening τ to capture more events) inflates Type I error and undermines credibility — pre-register or report sensitivity transparently.

Connections

Survival techniques connect directly to delayed-label learning for ML models, time-varying treatments in causal inference, and uplift/heterogeneous treatment effect estimation when outcomes arrive over time.