Pinterest wants to improve the perceived freshness and engagement of the home feed. Answer the following interview questions: 1. Define a practical metric for fresh content. Explain what counts as fresh, what user action or exposure should be measured, and what time window you would use. Discuss why you would choose that window instead of a shorter or longer one. 2. Discuss weaknesses of your freshness metric. What could it miss? How might heavy users, light users, creators with different posting frequencies, and different content categories affect the metric? 3. Pinterest is considering increasing the number of video pins shown in the home feed to increase user engagement. State the null hypothesis and at least two alternative hypotheses, including beneficial and harmful alternatives. 4. Suppose an A/B test is randomized at the user level for 14 days. Interpret the following results, including the meaning of the p-values and whether you would launch the change. Metric | Control | Treatment | Relative lift | p-value ---|---:|---:|---:|---: Users | 100000 | 100000 | - | - Home feed sessions per user | 5.00 | 5.03 | +0.6% | 0.08 Click-through rate | 4.00% | 3.92% | -2.0% | 0.03 Saves per 100 impressions | 1.20 | 1.26 | +5.0% | 0.01 Video watch time per user | 30.0 seconds | 33.0 seconds | +10.0% | <0.001 Hide or report rate | 0.50% | 0.53% | +6.0% | 0.04 7-day return rate | 42.0% | 41.8% | -0.5% | 0.20 Fresh impression share | 18.0% | 17.1% | -5.0% | 0.02 5. Explain how to calculate sample size for an A/B test. How do baseline variance, minimum detectable effect, significance level, statistical power, traffic allocation, trigger rate, CUPED, and multiple metrics affect power?