C++ Systems Programming For Infrastructure

What's being tested

Interviewers are probing practical mastery of writing high-performance, correct C++ system code for infrastructure: safe concurrency, predictable memory behavior, and observability under load. Expect to demonstrate applying the C++ memory model, correct synchronization (`std::atomic`, fences), resource ownership patterns (RAII, `std::unique_ptr`/`std::shared_ptr`), and performance engineering (cache locality, allocation strategies, profiling). They want crisp tradeoffs: simple correct design vs low-latency/high-throughput optimizations and how you'd validate them.

Core knowledge

C++ memory model (C++11+): understand sequential consistency, `memory_order_seq_cst`, `memory_order_acquire`/`release`, `memory_order_relaxed`, and when fences are necessary to enforce happens-before relationships for lock-free algorithms.
Atomics and lock-free primitives: `std::atomic<T>`, `compare_exchange_weak`/`strong`, ABA problem, and practical limits: lock-free for pointers/integers is common; complex structures usually need synchronization.
Mutexes and locking strategies: `std::mutex`, `std::shared_mutex`, and `std::unique_lock`; know coarse-grain vs fine-grain locking, lock striping, and deadlock-avoidance by lock ordering.
Ownership & lifetime: RAII, Rule of Five, `std::unique_ptr` for exclusive ownership, `std::shared_ptr` atomic refcounts cost ~20–40ns per operation; prefer `unique_ptr` + explicit sharing when throughput matters.
Undefined behavior (UB) traps: strict aliasing, iterator invalidation, data races (even read/write), use of `std::launder`/placement new; UB can invalidate reasoning and optimizations.
Memory/layout & locality: object packing/padding, `alignas`, `offsetof`; avoid false sharing by padding hot data to cache-line (typically 64 bytes); prefer SoA vs AoS for vectorized access.
Allocators and fragmentation: general-purpose `new`/`malloc` may be a bottleneck at high concurrency; use pooled allocators, thread-local arenas, or `jemalloc` for heavy-allocation workloads to reduce contention.
I/O and syscalls for infra services: use async multiplexing (`epoll`/`kqueue`), nonblocking sockets, and zero-copy (`sendfile`/`splice`) where appropriate to reduce context switches and copies.
Profiling, sanitizers, and observability: iterate with `perf`/`flamegraphs`, `clang-tidy`, `clang-asan`/`ubsan` for UB, `tsan` for data races, `valgrind` for memory errors; add `p99`/`p95` latency metrics and structured tracing.
Concurrency testing & correctness: deterministic unit tests are insufficient; use stress tests, fuzzers, model checkers, and death tests; log invariants and fail-fast on invariant violations.
Performance tradeoffs quantification: always attach numbers — e.g., switching `std::mutex` to lock-free atomics may reduce latency by X% but increase code complexity and risk; measure before optimizing.

Tip: For low-latency paths, prefer single-producer-single-consumer (SPSC) designs and per-thread buffers to avoid shared contention.

Worked example — "Implement a thread-safe LRU cache in C++"

First 30 seconds: clarify capacity, required concurrency level (single writer vs many readers), eviction policy ties, persistence, and TTLs. Frame success criteria: correctness (no races), throughput (ops/sec), and eviction latency. Skeleton answer pillars: data structures (hash map + doubly-linked list for recency), concurrency model (coarse `std::mutex` vs striped locks vs `std::shared_mutex` for readers), and memory management (store values in `std::unique_ptr` to avoid copies). Explicit tradeoff: a single global `std::mutex` is simplest and safe but serializes access — acceptable for small caches, not for high throughput; lock striping or per-bucket locks increase parallelism but complicate eviction across buckets. Mention implementation details: move semantics for values, careful handling of iterator invalidation when removing nodes, and constant-time eviction via list splicing. Close by saying: if more time, I'd add benchmarks vs real workload, a TTL background reaper, and consider a segmented LRU (multiple independent LRU shards) to reduce contention.

A second angle — "Design a high-throughput logging ring buffer"

This problem stresses the same foundations but with different constraints: single-producer-single-consumer (SPSC) vs multiple producers changes choice of primitives. For SPSC, a circular buffer with two `std::atomic<size_t>` indices works; use `memory_order_relaxed` for head/tail in tight loops and `memory_order_acquire/release` when crossing ownership. Avoid false sharing by padding indices to separate cache lines. If multiple producers are required, use `compare_exchange` on write position or a producer queue per thread to avoid contention. Also cover blocking vs spinning: use `futex`/`condition_variable`/`eventfd` to sleep when empty/full, and measure whether busy-waiting is acceptable for your latency targets.

Common pitfalls

Pitfall: Thinking `std::shared_ptr` is free — analysts often overuse `std::shared_ptr` without considering atomic refcounting cost; for hot paths replace with `std::unique_ptr`+copy-on-write or manual ref management.

Pitfall: Not asking about failure and memory constraints — ignoring eviction policies, TLS size, or crash-consistency can make an otherwise-correct design unusable in production.

Pitfall: Equating "passes unit tests" with correctness — missed concurrency bugs surface only under stress; always run TSan, long-running stress tests, and real-load profiling.

Connections

Interviewers may pivot to adjacent areas: memory allocators and how allocator design affects fragmentation and throughput, or networking stack choices (`epoll` vs async IO) for services. They might also ask about distributed consistency implications when a local cache is used across nodes.