Trigger Rules

Concepts covered: Trigger rules and branching

Before the scheduler moves a task to scheduled, it reads the task's trigger rule against the final states of every task directly upstream. The default rule is all_success: each upstream must have ended in success. Under it, 1 failed upstream puts the task in upstream_failed and 1 skipped upstream puts it in skipped, so a single failure near the top of a DAG paints the column below it orange without any of those tasks starting. The other rules change what counts as enough, and some of them stop waiting for the remaining upstream the moment their condition is met.

from datetime import datetime

from airflow.sdk import dag, task


@dag(schedule="@daily", start_date=datetime(2026, 11, 1), catchup=False)
def nightly_inventory():
    @task
    def pull_warehouse_counts():
        print("warehouse counts pulled")

    @task
    def pull_store_counts():
        raise ValueError("store feed empty")

    @task
    def merge_counts():
        print("counts merged")

    @task
    def publish_report():
        print("report published")

    @task(trigger_rule="one_failed")
    def page_oncall():
        print("a count feed failed")

    @task(trigger_rule="all_done")
    def drop_staging():
        print("staging dropped")

    warehouse = pull_warehouse_counts()
    store = pull_store_counts()
    [warehouse, store] >> merge_counts() >> publish_report()
    [warehouse, store] >> page_oncall()
    [warehouse, store] >> drop_staging()


nightly_inventory()

merge_counts keeps the default rule and needs both feeds green. page_oncall carries one_failed, so the scheduler evaluates it the instant either feed fails, while the other feed may still be pulling. drop_staging carries all_done, which waits for both feeds to reach a final state of any kind and then runs. A trigger rule is set per task, as a keyword on @task or on any operator, and it looks only 1 level up: publish_report reads the state of merge_counts and never sees the feeds behind it.

A twelfth rule exists for a task type covered later in the track; the 11 above cover every ordinary task.

• 4
• 2

One failed feed, 6 different cells

Play the run for the 1 November window, which appears at 00:00 on 2 November. Both feeds start at 00:01. pull_store_counts raises at 00:06 and, with 0 retries, goes to failed. The scheduler now evaluates the 3 tasks behind the feeds. page_oncall sees 1 failed upstream and starts at 00:06 while the warehouse feed is still running. drop_staging waits, because all_done needs every upstream final, and starts at 00:09 when pull_warehouse_counts succeeds. merge_counts sees a failure under all_success and is set to upstream_failed without a worker ever picking it up, and publish_report follows at the same moment, because an upstream in upstream_failed counts as failed.

About this lesson

This section is part of the Task Lifecycle: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.