Design a VM Provisioning Service, From the API Down to the Hypervisor Host
Company: Oracle
Role: Software Engineer
Category: System Design
Difficulty: hard
Interview Round: Onsite
Design a service that provisions virtual machines on a cloud compute platform. A customer asks for a VM of a given shape (number of virtual CPUs and amount of memory), built from a chosen operating system image and attached to a chosen network. The service finds a physical host with room for it, creates the VM on that host, and reports when it is running.
The report gives only the problem's name, but it indicates that the round expected low-level design detail and an understanding of how hypervisors and virtual machines work underneath. Be ready to go from the public API all the way down to what happens on the host.
```hint Accept now, finish later
Creating a VM involves several services and takes far longer than one API call should block. Decide what the create call returns immediately, and where the request's progress is recorded so that a crash halfway through neither loses it nor creates the VM twice.
```
```hint Two requests, one free slot
Ask what stops two concurrent requests from both claiming the last free capacity on the same host.
```
```hint Below the API
List what must exist on the physical host before the guest operating system can boot, and which component creates each piece.
```
### Clarifying Questions
- How many hosts and zones are in scope, how many create requests arrive per second at peak, and how quickly must a VM be running?
- Are shapes fixed or custom, and do some need special hardware such as GPUs or local NVMe disks?
- Is the boot disk a network-attached block volume or a disk local to the host, and how large are images?
- Which lifecycle operations are in scope besides create: stop, start, reboot, terminate, resize, live migration?
- Which placement constraints apply: zone or fault-domain choice, anti-affinity between a customer's VMs, per-tenant quotas?
- Does "running" mean the hypervisor has started the VM, or that the guest has finished booting and is reachable?
### What a Strong Answer Covers
- Requirements, scale estimates and the meaning of "running" agreed before the design
- An asynchronous create API with idempotency, backed by a durable per-VM state machine whose steps are safe to retry
- Capacity tracking and placement: filtering and scoring hosts, honoring constraints, and reserving capacity atomically so that no host is double-booked
- What the host agent and hypervisor do: CPU and memory allocation, hardware-assisted virtualization, the boot disk built from an image, the virtual NIC and network programming, and guest initialization
- A low-level design of the main components and their interfaces
- Failure handling: timeouts with unknown outcomes, compensation of partial work, fencing, and a reconciler that converges desired and actual state
- Scaling by zones or cells, image distribution, quotas, isolation between tenants, and observability
### Follow-up Questions
- A burst of a thousand create requests for one shape lands on a zone that is nearly full. How does placement stay correct and fast, and what does the customer see when capacity runs out?
- The host agent starts a VM but crashes before reporting back, and the workflow times out. How does the system converge without leaking capacity or running two copies of the VM?
- How would you cut the time to running for a popular image?
- How does a host keep one VM from slowing its neighbors, and what changes if you overcommit CPU or memory?
Overview: A system design question asking how to build a service that provisions virtual machines on a cloud platform, from the create API and capacity placement down to what the host agent and hypervisor do before a guest boots. It tests asynchronous workflows, idempotency, double-booking prevention, failure cleanup and virtualization fundamentals.
Read the full Oracle Software Engineer interview experience this question came from