Daniel (Weixiang) Wan

Hulu · Automated Data Recovery

A successful data job still needs a completeness check.

A downstream job can finish reading an hour while the upstream source is still adding records. The recovery flow I improved compares source and destination counts for that hour to identify the scope needing investigation and repair.

Read case study →
Job succeeded≠ complete dataSource rowsSame hourTarget rowsSame hourHourly comparisonFind mismatchAnomalous hoursReady to schedule