Transform clickstream with pandas sessionization
Overview
This question evaluates proficiency in time-series data manipulation, sessionization logic, timestamp imputation, robust ordering of out-of-order events, and scalable chunked processing using pandas or SQL-based techniques.
Given a pandas DataFrame events with columns [user_id:int, ts:str ISO8601 or NaT, url:str, server_log_ts:datetime], build 30-minute inactivity sessions per user: 1) Use server_log_ts to impute ts when ts is missing; 2) Robustly sort events per user with potentially out-of-order rows; 3) Define session_id when the gap > 30 minutes; 4) Compute, for each user, session_count, median_session_duration, and the 95th percentile of pages per session; 5) Ensure the solution works in streaming-sized chunks (cannot load all users into memory). Provide vectorized code sketches and explain correctness on edge cases (exactly-30-minute gaps, duplicated events, DST shifts).