## Design an SSH-Based Cloud Development Environment Design a cloud development environment that is accessed by SSH rather than through a browser. Users obtain isolated remote workspaces on hosts and submit build, test, or run tasks through a scheduler layered over those hosts. Clarify workspace persistence, task duration, required isolation, fleet size, startup latency, and whether an interactive shell and scheduled tasks may run concurrently in one workspace. ### Part 1 — Define Workspace and Access Contracts Specify APIs for creating, starting, stopping, inspecting, and deleting a workspace, plus the SSH connection flow and durable records. #### What This Part Should Cover - Stable user, workspace, host-allocation, session, and task identities. - SSH public-key or certificate authentication and short-lived authorization. - Persistent source volume separated from disposable compute. - Workspace lifecycle, quotas, idle timeout, and deletion semantics. ```hint Separate identity from host address A workspace should survive rescheduling even though the machine and SSH endpoint serving it can change. ``` ### Part 2 — Allocate Hosts and Establish SSH Sessions Design placement, host-agent communication, endpoint discovery, and the path from an authenticated SSH client to an isolated workspace. #### What This Part Should Cover - Versioned host capacity, health, image, and locality information. - Atomic or leased workspace reservations. - A gateway or signed routing record that avoids trusting a client-selected host. - Isolation of processes, filesystem, network, secrets, and host control APIs. ```hint Fence stale assignments A recovered scheduler must prevent an old host from continuing to serve a workspace after ownership moved elsewhere. ``` ### Part 3 — Schedule Build and Run Tasks Define the task API, queue, fairness, execution state machine, logs, cancellation, and interaction with the user's live workspace. #### What This Part Should Cover - Idempotent submission and immutable task specifications. - Queued, leased, running, and terminal states with attempt identity. - Per-user fairness, resource limits, and bounded backlog. - Output and log persistence independent of one worker process. ```hint Interactive and batch work share capacity Reserve enough resources for a usable shell while preventing one user's scheduled tasks from consuming the entire host. ``` ### Part 4 — Recover and Operate the Fleet Handle host loss, scheduler failover, SSH disconnect, task-launch ambiguity, workspace checkpointing, overload, and observability. #### What This Part Should Cover - Reconciliation of desired allocation with host-agent observations. - Idempotent launch and fencing before retrying on another host. - Explicit durability and recovery-point objectives for workspace files. - Metrics for startup, session reliability, queue delay, utilization, and isolation failures. ```hint A disconnect is not a stop command The SSH transport can fail while the workspace and its tasks continue, so lifecycle state cannot depend on one socket. ``` ### What a Strong Answer Covers - Treats SSH access, workspace lifecycle, and task scheduling as connected but separate concerns. - Uses durable identities, leased placement, and fenced host agents. - Preserves workspace data while allowing compute rescheduling. - Includes multi-tenant isolation, fairness, failure reconciliation, and auditable access. ### Follow-up Questions 1. How would a user reconnect after the original host fails? 2. What happens to an interactive process when a workspace is migrated? 3. How would scheduled tests read an uncommitted working tree safely? 4. Which controls prevent an SSH user from attacking the host or another workspace?