Project Deep Dive: Kafka Streaming, Redis Caching, and a Go Edge API Gateway

This technical screen opened with about ten minutes of intense resume questions before the coding problem. The interviewer concentrated on two items from the candidate's resume and asked for detail and trade-offs on each: - backend work from a previous internship built on a Kafka streaming architecture with Redis caching; - a project in which the candidate built a distributed edge API gateway in Go. Practice answering the interviewer's probes below as if they were asked about your own work. If your resume lists different systems, answer the same probes about the closest system you actually built; they carry over to any streaming pipeline, cache or latency-sensitive service. ### Clarifying Questions - Should I start with the Kafka pipeline's end-to-end flow, or go straight to the part you care about most, such as partitioning, delivery guarantees or the Redis layer? - For the gateway, are you more interested in the language decision itself or in the latency work that followed from it? - Do you want production numbers, or is a design-level explanation acceptable where I only have estimates? ### Part 1 — The Kafka streaming architecture "Walk me through the streaming architecture from your internship." Describe what produced the events, how topics and partitions were laid out and keyed, which services consumed them, and what delivery and ordering guarantees the pipeline provided. Be ready for: why Kafka, and what happened when a consumer crashed or a message could not be processed. ```hint Start from the key Ordering, consumer parallelism and hot spots all follow from how events were keyed to partitions. Know what your key was and why you chose it. ``` #### What This Part Should Cover - The end-to-end flow, the partitioning scheme and the consumer-group layout - Delivery and ordering guarantees, and how duplicates and failing messages were handled - How the pipeline's health was observed, and why Kafka rather than a simpler queue ### Part 2 — The Redis caching implementation "How did you use Redis?" Explain what was cached and why, how reads and writes went through the cache, how entries expired or were invalidated, and what happened when the cache disagreed with the database or Redis was unavailable. ```hint Trace one write Follow a single update from the database to the cache and find the window in which a reader could still see the old value. ``` #### What This Part Should Cover - The caching pattern, key design and expiry policy - Invalidation and the staleness window it leaves - Behavior under a burst of misses, a hot key and a Redis outage ### Part 3 — Why Go for the edge API gateway "Why did you build the gateway in Go?" Justify the choice against the gateway's workload, name the alternatives you considered, and say what Go cost you. ```hint Describe the work per request first Say what the gateway spends its time and memory on for each request before naming language features; the choice should follow from that. ``` #### What This Part Should Cover - The workload properties that drove the choice - Specific Go properties that helped, and those that hurt, such as garbage collection - A credible comparison with at least one other language, or with extending an existing proxy ### Part 4 — Managing latency in a distributed edge gateway "How did you manage latency?" Explain how you measured latency across the gateway's nodes, where the time went, and which changes reduced it, especially at the tail. ```hint Break the request into hops Account separately for the time before a request reaches the gateway, the time inside it, and the time between it and the upstream services, and say how you measured each. ``` #### What This Part Should Cover - Measurement: percentiles, a per-hop breakdown and tracing - Concrete techniques for each hop, with their costs - Protecting tail latency under load and when an upstream is slow or failing ### What a Strong Answer Covers - Clear ownership: what you built yourself versus what the team or platform provided - Each decision tied to a requirement, with the rejected alternative named - Numbers you can defend, such as throughput, latency percentiles or cache hit rate, labeled as estimates where they are estimates - Failure modes, and what you would change with hindsight - Concise answers that fit a ten-minute window, with more depth available on request ### Follow-up Questions - If traffic grew tenfold, which part of the Kafka pipeline would break first, and which signal would warn you? - Could the Kafka stream keep the Redis cache consistent instead of invalidating it on the write path? What would change? - The gateway runs in several regions. How would you enforce a global per-client rate limit without a cross-region call on every request? - What would convince you to rewrite the gateway in another language, or to replace it with an existing proxy?