RL Agent Automates ETL Pipeline Failure Remediation

Anna Marie Benzon, a PhD candidate in Artificial Intelligence from the University of the Philippines Diliman, presented a novel approach to automating the detection and remediation of ETL pipeline failures. Her work introduces a Reinforcement Learning (RL) agent designed to enhance the reliability and efficiency of data processing workflows. Traditional methods often involve extensive manual log inspection, schema tracing, and delayed dashboards, leading to a Mean Time To Recovery (MTTR) of approximately 2.5 working days. Benzon's solution aims to transform this reactive debugging process into intelligent recovery, making routine ETL failures diagnosable, explainable, and recoverable in minutes.

The Problem: The Cost of ETL Failures

Cloud ETL jobs frequently break due to a variety of issues, including late or unavailable source data, schema drift, datetime parsing incompatibilities, null-rate spikes, type changes, and unknown runtime errors. The manual process of identifying, diagnosing, and fixing these failures is time-consuming and resource-intensive. Benzon highlighted that the manual workflow typically involves failure, inspection of logs, diagnosis, repair, rerun, and validation, a cycle that can take considerable time. The core challenge is not just detecting a failure, but doing so effectively and remediating it within operational boundaries that a human team can trust.

The Solution: An End-to-End RL Pipeline Health Agent

Benzon detailed the architecture of an end-to-end RL pipeline health agent. The system monitors ETL jobs, diagnoses failures, scores their operational risk, decides on an appropriate action, ensures safety, acts to remediate, and validates the outcome. The architecture involves components like AWS CloudWatch Logs, Amazon EventBridge, AWS Lambda, a data catalog, and the RL agent itself, interacting with services like AWS Glue and Amazon S3. The RL agent receives state information such as failure category, risk level, retry count, drift severity, and data quality conditions. Based on this state, it selects an action from a set including retry, coerce schema, rollback, quarantine, escalate, or log. The agent utilizes tabular Q-learning, a small, interpretable state space, low-memory inference, and inspectable Q-values for each decision, making it a practical and understandable solution.

Intelligence Layers and Safe Autonomy

The proposed system features an intelligence layer composed of three distinct components: Deterministic Anomaly Rules, a Q-Learning Decision Policy, and a Safety Override. Deterministic anomaly rules are used for observable facts like schema drift, null spikes, and type changes. The Q-learning decision policy handles more contextual actions such as retries, schema coercion, rollbacks, quarantines, escalations, and logging. A critical safety override layer ensures that in cases of critical anomalies or unsafe passive actions, the system escalates immediately. This layered approach ensures that while the RL agent learns and proposes actions, its autonomy is bounded by predefined safety constraints, and every decision generates an audit record.

Evaluation and Reproducibility

The research emphasizes reproducible evaluation, with the system designed for independent reproduction. It utilizes a generalized AWS Lambda-style architecture, synthetic schemas, records, logs, and incidents, avoiding the use of production data or infrastructure identifiers. Four controlled experiments were conducted, and robustness was checked across 30 runs with various seeds. The results, reported with 95% confidence intervals, demonstrate significant improvements. On a controlled synthetic benchmark, the RL-based anomaly detector achieved a precision of 1.000, a recall of 0.800, and an F1 score of 0.889. More importantly, the successful case resolution time was reduced to approximately 5.2 minutes, a substantial improvement over the manual process. The RL agent also showed a simulated success rate of 74.63% and a non-escalation rate of 88.63%.

Key Findings and Future Directions

The evaluation revealed that the deterministic rules performed comparably to the RL policy in this benchmark, with the safety override being a key contributor to reducing non-escalation rates. The research indicates that while the RL agent provides inspectable learned policies, its performance in this specific benchmark did not significantly outperform deterministic rules. However, the combination of structured decision logic and external guardrails produced most of the reliability. Current limitations include the reliance on synthetic scenarios and the agent being failure-triggered rather than a pre-failure predictor. Future work will focus on validating the system with real-world production data, expanding the state space, and enabling online learning with appropriate operational approval gates.

Benzon concluded by stating that a practical self-healing pipeline does not necessarily require a giant model but rather needs bounded authority, reproducible evidence, and the discipline to escalate when uncertainty exists. The work showcases how smaller, well-defined AI components can deliver significant operational value in complex data pipelines.