Design streaming error-log counting with moving average

Q: Design streaming error-log counting with moving average

This question evaluates understanding of real-time stream processing concepts such as event filtering, stateful per-service counting, sliding-window moving averages, late-event handling, scalability, and fault-tolerant state management for monitoring and alerting.

Q: How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

Question

Design a core component in a streaming system:

Input:

Multiple upstream services continuously emit log events.
Each event includes at least: service_id , timestamp , log_level , message .

Tasks:

Filter and output only error logs .
Maintain real-time per-service error count .
Maintain a moving average of error count per service over a sliding time window.
Trigger an alarm when a service’s error rate/moving average crosses a threshold.

Describe the architecture, state management, windowing approach, and how you handle late events, scale, and fault tolerance.

Design streaming error-log counting with moving average

Quick Overview

Solution

Comments (0)