Runs That Wait on Past Runs

Concepts covered: Catchup and backfill

An inventory rollup table holds a running stock count per warehouse: each hourly run reads the count the previous run wrote and adds this hour's movements. If the run for 01:00 fails and the run for 02:00 goes ahead, 02:00 adds its movements to the count from 00:00, and every later hour inherits the gap. The run for 02:00 has to wait until 01:00 has finished, and finished well.

depends_on_past=True is a task setting, set on 1 task or through default_args for all of them. Before the scheduler starts an instance of that task, it looks at the same task in the DAG's previous run, the run with the latest logical date before this one. If that instance reached success or skipped, this one may start. If it failed, or has not finished, this one waits with no state, and the scheduler checks again on every loop. The first run of the DAG has no previous run and starts without waiting, and the tasks without the setting in a later run proceed as usual.

from datetime import datetime
from airflow.sdk import dag, task
from airflow.timetables.interval import (
    CronDataIntervalTimetable as Cron)

@dag(schedule=Cron("@hourly", "UTC"),
     start_date=datetime(2026, 11, 10),
     catchup=False)
def inventory_rollup():
    @task
    def extract():
        return 1

    @task(depends_on_past=True)
    def rollup(n):
        return n

    rollup(extract())

inventory_rollup()

Recovery runs in order

The failed instance in the 01:00 run is the only thing holding the chain. Fix the cause, then clear that one instance from the grid, an action the Task Lifecycle lessons cover in depth; here it means the instance runs again. When it succeeds, the 02:00 run's rollup becomes eligible on the next scheduler loop, the 03:00 run's rollup becomes eligible only once the 02:00 one has succeeded, and so on down the grid. Each rollup starts after the one before it finishes, so 6 blocked hours take 6 rollup durations to drain. Marking a later instance success by hand lets the run after it start, and leaves the hole in the data that the setting exists to prevent.

A 6-hour outage of the rollup store

Say the store the rollup writes to is down from 02:00 to 08:00. The rollup created at 02:00 fails; the 6 runs created from 03:00 to 08:00 arrive on time, their extract instances succeed, and their 6 rollups wait. No other task fails, since the tasks after each waiting rollup never start. At 08:10 the 02:00 rollup is cleared and succeeds, and the 6 waiting rollups run back to back, oldest first, each reading the count the previous one wrote. With depends_on_past=False, the 6 rollups that started inside the outage would have failed, the one created at 08:00 would have succeeded on a count missing 6 hours, and clearing the failed 6 would run them in parallel, each reading whatever count happened to be there.

• A column of empty cells under a failed one in the same task row is a depends_on_past chain
• Runs stuck in running with every other task green are waiting runs, not slow ones
• The oldest failed cell in the row is the one to fix; the rest drain by themselves

depends_on_past=True

• Per task
• Checks the outcome of the previous run's instance
• A failure holds every later run's instance of that task
• Other tasks in later runs still proceed

max_active_runs=1

• Per DAG
• Counts runs still in progress, whatever their outcome
• A failed run is finished, so the next run starts
• Nothing in the next run proceeds until the whole previous run ends

wait_for_downstream=True tightens the rule by 1 step: the task waits until the previous run's instance of itself and of every task immediately downstream of it have reached success or skipped. It covers the case where the previous run's rollup succeeded but its publish did not, and this run's rollup would otherwise overwrite rows publish has not read yet. Setting it turns depends_on_past on for that task too.

About this lesson

This section is part of the Scheduling: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.