Troubleshoot a Midnight Web Server Outage
Company: Meta
Role: Site Reliability Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Technical Screen
You are on call for a Linux-based web application that lets customers submit orders and receive confirmations. In the middle of the night, you receive an alert that the service is down. The incident is still ongoing, all users are affected, there was no recent code deployment, and traffic volume has not changed.
You do not have dashboards or metrics. You can only investigate through Linux commands and by checking logs.
During debugging, you discover the following:
- Application and system logs show that one API is returning HTTP 500 errors.
- That API fails while trying to write data to the database.
- The server disk is not full.
- The main database storage is not full.
- A request queue is also not backed up.
Given these findings:
1. Explain how you would systematically troubleshoot the outage.
2. Identify the most likely root cause.
3. Describe how you would confirm it.
4. Describe both the immediate mitigation and the long-term prevention steps.
Quick Answer: This question evaluates a candidate's incident response, systems debugging, and root-cause analysis skills, focusing on log-driven investigation, Linux command-line troubleshooting, and diagnosing application-to-database failures.