Design a Chat Analytics Backend
Quick Overview
This question evaluates a candidate's competence in designing large-scale analytics backends, covering data ingestion, event and metric schemas, near-real-time and historical analytics, metric computation and slicing, data freshness and deduplication, privacy and access control, operational monitoring, and the API/query layer.
Design the backend of an internal analytics system for a conversational AI product similar to ChatGPT. The goal is to power an analytical metrics dashboard used by product managers and backend engineers.
The dashboard itself and the LLM serving stack are out of scope. Focus on the data platform and backend services that collect data from existing production databases and service logs, compute product and reliability metrics, and serve those metrics to the dashboard.
Your design should address:
- How to ingest data from existing services without overloading primary databases
- What event and metric schemas you would use
- How to support both near-real-time metrics and historical analysis
- How to compute common metrics such as daily active users, number of conversations, request volume, latency percentiles, token usage, error rates, and feature adoption
- How to slice metrics by dimensions such as time, model version, region, platform, and user segment
- Data freshness, correctness, deduplication, and backfills
- Privacy, access control, and operational monitoring
- The APIs or query layer that a dashboard backend would call
Assume the system must support large-scale traffic and multi-month historical retention.