CI/CD Pipelines

What's being tested

These interviews test whether you can turn an ambiguous delivery requirement into a reliable, secure CI/CD pipeline with explicit boundaries, interfaces, and failure handling. You must connect source control, build execution, artifact storage, Kubernetes deployment, authorization, observability, and rollback rather than describing isolated tools. For Salesforce, the underlying skill is operating software safely at large scale and across many teams, where tenant isolation, auditability, availability, and controlled change are as important as deployment speed.

Core knowledge

• A robust pipeline separates continuous integration from continuous delivery: validate every commit, produce an immutable artifact once, then promote that exact artifact through environments. Rebuilding during production deployment creates non-reproducibility and undermines rollback.

• Model the pipeline as stages: source trigger, dependency resolution, compilation, unit and integration tests, security checks, artifact publication, deployment, verification, and promotion. Each stage should have a clear contract, timeout, retry policy, and durable status.

• Prefer immutable artifacts identified by content digest, such as an OCI image digest, rather than mutable tags like latest. Store binaries and metadata in a registry such as Artifact Registry, ECR, or Harbor; retain provenance, commit SHA, dependency lockfile, and build environment.

• Kubernetes separates a managed control plane from worker nodes. The API server, scheduler, controller manager, and etcd maintain desired state; nodes run kubelet, a container runtime, and networking components. Managed control planes reduce operational burden but do not secure workloads automatically.

• Use layered Kubernetes security: least-privilege RBAC, separate namespaces, NetworkPolicy, restricted pod security settings, non-root containers, read-only filesystems, resource requests and limits, and admission controls such as OPA Gatekeeper or Kyverno.

• Never place long-lived cloud credentials in repository secrets or container images. Use short-lived workload identity, for example OIDC federation from the runner, and store only necessary secrets in Vault or a cloud secret manager; audit every privileged action.

• A deployment controller should reconcile desired and observed state. GitOps tools such as Argo CD provide an auditable model, while direct API deployment can be simpler and faster; explicitly compare drift detection, access control, rollback ergonomics, and operational complexity.

• Safe rollout strategies include rolling updates, blue-green deployments, and canaries. A canary limits blast radius, but requires health checks and automated abort thresholds; blue-green simplifies rollback but temporarily requires capacity for two versions.

• Define readiness, liveness, and startup probes correctly. Readiness removes an unhealthy instance from traffic; liveness restarts a stuck process; startup prevents slow initialization from being killed. A liveness probe that checks a downstream database can cause cascading restarts.

• For CI-job throttling, a token bucket supports bursts while enforcing an average rate: tokens accumulate up to capacity $B$ at rate $r$, and each job consumes tokens. A distributed implementation needs atomic state, commonly Redis Lua scripts, plus lease expiry so crashed runners do not hold capacity forever.

• Rate limiting must define its scope: per repository, organization, runner pool, priority class, or global system. Fair scheduling often combines quotas with weighted queues; strict FIFO can let one large organization starve small teams, while purely per-tenant limits waste idle capacity.

• Make retries safe with bounded exponential backoff and jitter. Retry transient registry, API, or node failures, but do not blindly retry compilation or deployment steps that may have side effects; use idempotency keys, operation IDs, and reconciliation to prevent duplicate releases.

• Pipeline reliability is measurable through deployment frequency, lead time, change-failure rate, and time to restore, but system design should also track queue wait time, runner utilization, p95/p99 stage duration, flaky-test rate, rollback time, and failed-admission counts.

Worked example

For “Design secure Kubernetes with CI/CD”, a strong candidate first asks about deployment frequency, number of teams and clusters, compliance requirements, workload sensitivity, cloud provider, availability targets, and whether teams share runner infrastructure. They should declare assumptions such as multiple environments, immutable container images, managed Kubernetes, and a requirement that untrusted pull requests cannot obtain production credentials. The answer can then be organized around pipeline stages and artifact provenance, cluster and workload isolation, identity and secret management, and rollout observability with rollback. The candidate should explain that CI produces and signs an image, scans it, stores it by digest, and promotes that same digest through environments rather than rebuilding. They should distinguish control-plane responsibility from node and application responsibility, then describe RBAC, admission policy, network boundaries, pod hardening, and audit logs. One explicit tradeoff is whether to use GitOps reconciliation for stronger drift detection and auditability or direct deployment APIs for a simpler, lower-latency path. They should close by noting that they would next validate disaster recovery for etcd and registries, model multi-region failure, and run an incident exercise involving compromised build credentials.

A second angle

For “Implement a Rate Limiter for CI Jobs Rather Than for API Requests”, the central concern shifts from cluster hardening to protecting finite runner capacity and keeping scheduling fair under bursty workloads. Clarify whether jobs are queued or rejected, whether limits are global or per organization, how much burst capacity is acceptable, and what happens when a runner disappears. A token bucket with atomic distributed updates handles average throughput and bursts, while a durable priority queue and leases handle fairness and crash recovery. The important tradeoff is between strict per-tenant isolation, which protects noisy neighbors, and high global utilization, which may leave capacity idle during quiet periods.

Common pitfalls

Analytical mistake. A tempting answer says “use Kubernetes and add a rolling deployment,” but never defines artifact identity, authorization boundaries, health signals, or rollback conditions. A stronger answer traces one commit through immutable publication, policy validation, deployment, verification, and recovery.

Pitfall:

Communication mistake. Jumping immediately into Jenkins, GitHub Actions, or Argo CD makes the design sound tool-driven rather than requirement-driven. State assumptions first, organize the response into four or five pillars, and explain which properties matter independently of the vendor.

Pitfall:

Depth mistake. Saying “use a distributed rate limiter” or “secure the cluster with RBAC” is incomplete. Explain the algorithm, atomicity, lease expiry, quota scope, service-account permissions, admission policies, secret flow, and behavior when the control plane, registry, or runner pool is unavailable.

Connections

The interviewer may pivot to distributed locking, work queues and fair scheduling, supply-chain security, observability and SLOs, or disaster recovery. Be prepared to connect deployment safety with idempotency, eventual consistency, incident response, and multi-region architecture.

Further reading

• SLSA — A practical framework for build provenance and software supply-chain integrity.

• Kubernetes Documentation: Pod Security Standards — Concrete baseline controls for restricting workload privileges.

• The Twelve-Factor App — Useful principles for configuration, disposability, logs, and deployable services.