Tech Screen (60 minutes): SQL, statistics knowledge, A/B testing.
Case Study (60 minutes): business case discussion and problem solving.
HM (60 minutes): case study + behavioral questions.
XFN (60 minutes): communication style, cross-team collaboration, deep dive into past projects.
HRBP (30 minutes): motivation for job hunting, culture fit, growth potential.
Usually the first two rounds are scheduled together, but whether the later interviews continue depends on how the earlier rounds go.
What the first round actually looked like
About 50 minutes, in English:
Both sides introduce themselves → walk through a past project, with follow-ups → 3 SQL questions → an A/B testing case → my questions for the interviewer.
There was no separate test on statistics formulas. Experimentation knowledge was tested through my project and the case.
1. Past project
I introduced one project I had done: the business background, my own contribution, the method, and the result. Depending on the project, they may follow up on experiment design, the unit of randomization, metric choice, selection bias, observation window, and so on.
2. SQL: the Ads Manager campaign creation flow
The setup is that advertisers create a campaign through three pages: objective → adgroup → creative → submit.
The event table is ttam_logger:
date STRING date partition, format YYYYMMDD
timestamp TIMESTAMP event time
advertiser_id BIGINT advertiser ID
event STRING enter / next / back / submit
page STRING objective / adgroup / creative
There were 3 questions:
- Count the total number of submits on a given date. I confirmed whether they meant event count or distinct advertisers, and the interviewer said to go by event count.
- Compute the overall submit rate for a given week. The denominator needed clarifying; I confirmed on the spot that it should be the number of enter events, so the rate is submit count / enter count.
- Compute each advertiser's total time spent on the site on a given date. The maximum inactivity gap is 30 minutes, so you can't just use
MAX(timestamp) - MIN(timestamp). My approach was to order by advertiser and time, useLAGto get the previous event's time, and then add up the gaps between adjacent events that are no more than 30 minutes. A gap over 30 minutes counts as the advertiser having left and is not counted. The interviewer was fine with this approach and said I did not need to handle events crossing midnight.
The SQL did not have to be run, but I had to write the code and explain the logic.
3. Case: how would you evaluate an in-platform video editing feature?
Background: when an advertiser uploads a video and there is a problem such as the resolution, they used to have to leave the platform, fix it offline and upload it again. Now Ads Manager wants to add an in-platform video editor so advertisers can fix it directly on the platform.
The question: how would you evaluate the impact of this new feature?
The discussion and follow-ups included:
- How would you design the experiment?
- Should you randomize by campaign, by advertiser, or by the users who use the tool?
- What would you pick as the primary metric and the guardrail metrics?
- How would you measure the improvement in the campaign submission flow?
- How much traffic should go to Treatment and Control, and why would you or wouldn't you use 50/50?
The interviewer made it clear that the final business goal is to increase revenue by helping advertisers get through campaign creation more smoothly.
The last follow-up: after the experiment ends, only 2% of treatment advertisers actually used the new feature, and the overall lift is close to 0. Would you still recommend launching?
Note that the 2% here is the feature usage rate, not a revenue lift. You need to separate making the feature available, actually using it, and what the overall experiment result means when usage is that low.
Discussion
Loading comments…