Hulu · Streaming Reliability

Partition load, consumer lag, and batch time help locate the bottleneck.

I compare Kafka partition load, consumer lag, and Spark batch duration to test whether the signals support the same cause. That gives the team a specific workload to investigate before choosing a tuning change.