Design a Test Orchestration System
Company: Axon
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
Design a test orchestration service for an engineering organization.
The service should let users define different kinds of tests, schedule when they should run, execute them on available compute, and track their results. The prompt is intentionally open-ended: clarify requirements, state assumptions, and propose a reasonable architecture.
Address at least the following:
- How tests are defined, versioned, and configured.
- How users trigger tests manually or schedule recurring test runs.
- How the system dispatches work to executors or workers.
- How execution status, logs, artifacts, and results are stored and queried.
- How the system handles failures, retries, concurrency limits, and scaling.
- What APIs and data models you would expose.
Overview: This question evaluates system design and backend engineering competencies, including distributed orchestration, scheduling, scalability, reliability, operational monitoring, API design, and data modeling for a test orchestration service; it is categorized under System Design and related domains such as distributed systems and CI/CD platforms.