Use Metrics and Logs to Establish a Production Root Cause

Explain how you use metrics and logs to investigate the root cause of a production problem. Walk through a concrete incident or an explicitly hypothetical example and show how you move from a symptom to evidence for a cause. ### Constraints The specific production symptom and available instrumentation are not prescribed. State your example and what telemetry exists. Additional tools such as traces may be proposed, but do not assume they are already available. Avoid treating a correlation or a single error message as conclusive proof. ### Clarifying Questions - What changed in user-visible success rate, latency, or throughput, and when? - Which services and request paths are affected, and what changed before the onset? ```hint Narrow the affected population Compare failing and healthy requests by time, release, endpoint, instance, and dependency before collecting more undirected logs. ``` ### What a Strong Answer Covers - Impact and time-window definition, mitigation, and evidence preservation. - Metrics that locate a bottleneck and logs that test specific explanations. - Correlation identifiers, telemetry limitations, and causal validation. ### Follow-up Questions - What would you do if the relevant logs were missing or sampled? - How would you distinguish a downstream failure from a retry storm created by your own service?