Design a Multi-Tenant CI/CD System Triggered by Git Pushes
Context and Assumptions
-
An internal service sends a push event for each Git push containing: tenant ID, repository ID, and repository state (e.g., commit SHA, branch).
-
Each repository defines a workflow in a single YAML file at a fixed path (for example, .ci/workflow.yaml).
-
A workflow is a sequence (or DAG) of jobs; each job has steps/commands and may depend on others.
-
Users must be able to see live job logs and execution status while runs are in progress.
Requirements
Describe the overall architecture and these components:
-
Event intake and trigger handling.
-
Workflow YAML retrieval, parsing, and execution planning.
-
Orchestration and job scheduling.
-
Execution/runners and isolation model.
-
Live log and status streaming to users.
-
Storage of run metadata, logs, and artifacts.
-
Cross-cutting: tenant isolation, fairness, and fault tolerance.
State design choices, key data flows, and how the system scales.