Pluralsight – Effective Workflow Management with Apache Airflow

Manage and monitor workflows using Apache Airflow. This course will teach you how to schedule tasks with cron and timedelta, handle retries, and debug task execution using the Airflow UI for resilient, reliable pipelines.
Handling workflows without proper scheduling, monitoring, and failure recovery can lead to missed tasks, delayed pipelines, and frustrating debugging. Without visibility and resilience, even simple workflows can become unreliable at scale. In this course, Effective Workflow Management with Apache Airflow, you'll gain the ability to build reliable, scheduled, and observable workflows using Airflow's scheduling, retry, and monitoring capabilities. First, you'll explore how to schedule workflows using cron expressions and timedelta, and understand how settings like start_date, catchup, and depends_on_past affect execution. Next, you'll discover how to handle task failures by configuring retries, setting delay strategies, and managing failed runs directly from the UI. Finally, you'll learn how to monitor and debug task execution using Airflow's interface—reviewing logs, inspecting task states, and understanding execution flow through graph and tree views. When you're finished with this course, you'll have the skills and knowledge of workflow scheduling and resilience in Apache Airflow needed to build production-ready, self-healing data pipelines.
Homepage

https://app.pluralsight.com/library/courses/apache-airflow-workflow-management-effective/table-of-contents