Implement a bootstrap confidence interval and a permutation test in Python

In a data science onsite coding round, the interviewer provides a dataset and asks you to implement a bootstrap and a permutation test in Python. The code must actually run on the data without bugs, not just describe the logic, and the interviewer adds follow-up coding requests after the main tasks. The report does not describe the dataset or the exact wording of the tasks. This version assumes a pandas DataFrame `df` with one row per user and two columns: `group`, either `"control"` or `"treatment"`, and `value`, a numeric per-user metric such as sessions in a week. ### Clarifying Questions - May you call ready-made routines such as `scipy.stats.bootstrap` or `scipy.stats.permutation_test`, or must you write the resampling yourself? - Is each row an independent user, or can a user contribute several rows? - Can `value` be missing, and if so, should those rows be dropped? - Should results be reproducible from a random seed? ### Part 1 — Bootstrap confidence interval Write a function that returns a bootstrap percentile confidence interval for a statistic of one sample, then use it for the mean of `value` in the treatment group. ```python def bootstrap_ci(values, stat_fn=np.mean, n_boot=10_000, alpha=0.05, seed=None): """Return (point_estimate, (lower, upper)).""" ``` ```hint Size and replacement Think about how large each resample must be, whether it is drawn with or without replacement, and how an interval is read off the collection of resampled statistics. ``` #### What This Part Should Cover - Resamples of the right size drawn the right way, and the interval taken from the resampled statistics - Efficient computation that stays within memory, with a seed for reproducibility - Input validation and missing values ### Part 2 — Permutation test Write a function that tests whether the mean of `value` differs between treatment and control with a two-sided permutation test, returning the observed difference in means and the p-value. ```python def permutation_test(x, y, n_perm=10_000, seed=None): """Return (observed_difference, p_value) for mean(x) - mean(y).""" ``` ```hint What the null lets you do If the group labels had no effect on `value`, they would be interchangeable. Think about what that allows you to shuffle, and what you compare the observed difference with. ``` #### What This Part Should Cover - The null hypothesis and why it justifies shuffling group labels - A correct two-sided p-value from the permutation distribution, including how it handles a finite number of random permutations - Efficient computation and edge cases such as empty or very unequal groups ### What a Strong Answer Covers - Code that runs end to end on the given data, with a fixed seed and sensible defaults - Correct resampling schemes: with replacement within a sample for the bootstrap, label shuffling without replacement for the permutation test - Vectorized or batched computation that stays within memory - Handling of missing values, tiny or unequal groups, and users with several rows - Interpretation: what the interval and the p-value mean, and when each method is appropriate ### Follow-up Questions - Bootstrap a confidence interval for the difference in means between treatment and control. How does the resampling change? - Each user can contribute several rows. How do you change both functions? - How would you check that your permutation test has the right false-positive rate? - When the bootstrap interval and the permutation p-value disagree about significance, what could explain it?