Design the control plane for a cloud compute service like Azure

Design the control plane for a cloud compute service such as Azure's virtual machines: the management layer that accepts customers' requests, decides where their virtual machines run, and keeps every machine in the state the customer asked for. The report gives only the one-line prompt, so settle the scope with the interviewer first. In its usual form, customers call an API to create, start, stop, resize and delete virtual machines; the control plane validates and records each request, chooses a physical host with enough capacity, tells that host what to run, and tracks each machine's actual state until it matches the request. ```hint Desired state versus actual state Decide where the customer's intent is recorded, and how the system keeps driving hosts toward it when a step fails halfway through. ``` ```hint Running machines come first Think about what must keep working for virtual machines that are already running while the control plane itself is down or being upgraded. ``` ### Constraints and Clarifications - Not given: the scale (regions, hosts, virtual machines, request rate), the operations in scope, and how fast provisioning must be. Ask, or state your assumptions. ### Clarifying Questions - Which operations are in scope: create and delete only, or also start, stop, resize and live migration? - Is the design for one region, or global with regions that must fail independently? - How many hosts and virtual machines per region, and how many API requests per second? - How fast must a create request complete, and is it synchronous or a long-running operation that the client polls? - Which placement rules apply: availability zones, fault domains, spreading one customer's machines apart, special hardware such as GPUs? ### What a Strong Answer Covers - Scope, scale estimates and an API with long-running operations and idempotent retries - A durable record of desired state and a reconciliation loop that drives hosts toward it - A placement component that respects capacity and fault-domain constraints under concurrent requests - Host agents and the communication pattern between the control plane and a large fleet of hosts - Failure handling: partial failures, retries, host loss, and control-plane outages that leave running machines untouched - Regional isolation, safe rollout of the control plane itself, and observability ### Follow-up Questions - A host stops sending heartbeats. How do you tell a dead host from a network partition, and what happens to its virtual machines? - Two create requests race for the last free capacity on one host. How do you prevent overcommitting it? - How do you roll out a new version of the host agent across the fleet without causing an outage? - A customer retries a create request after a timeout. How do you make sure only one virtual machine is created?