Scheduling: Beginner

A scheduled run of a DAG answers one question: what happened in this slice of time? The slice has a start and an end, and Airflow calls it the run's data interval. A DAG that runs once an hour owns 1 hour per run, and the tasks inside that run read the rows that landed within that hour and nothing outside it. 3 time values ride along with every run: data_interval_start, data_interval_end and logical_date. For every scheduled run logical_date equals data_interval_start, so the run is named after the moment its window opened; runs triggered by hand, covered in the last section, follow a different rule.

Watch where each cell appears in the animation above: the run named 00:00 lights up at 01:00, the run named 01:00 lights up at 02:00, and so on down the grid. The file names a start_date of midnight and a schedule of 1 hour, and Airflow lays the hours end to end from there. Nothing in the file states when a run starts; that follows from where its window ends.

This gives you a way to read any moment on the clock. At 11:40 on 4 November, the runs that exist are the ones whose windows closed at or before 11:40: the runs named 10:00, 09:00 and every hour back to start_date. The 11:00 window is still open, so there is no run named 11:00 yet, and there will be none until 12:00. A grid that ends with the 10:00 column at 11:40 is a grid that is exactly up to date.

Every time in this lesson is UTC, which is what tz="UTC" on each start_date declares. Other time zones change where the boundaries fall and are covered in a later lesson.

A task gets its run's window by asking for it. With TaskFlow, name a parameter data_interval_start or data_interval_end and Airflow fills it with the run's value when the task starts, so the same function reads a different hour in every run with no change to the file. Compare that with datetime.now() called inside the task, which returns the moment the task happened to execute, 10:03 in one run and 10:07 in a slower one, and returns something else again if the task is run a second time.

Naming the run by its window is what makes a second run of it meaningful. Run scheduled__2026-11-04T09:00:00+00:00 again a week later and it is handed the same 2 values, counts the same hour and writes the same answer. The clock time it ran at has moved by 7 days and nothing it reads has moved at all. A task that trusted the clock instead would count a different hour on every attempt.

schedule takes 4 kinds of value. A timedelta lays windows of 1 fixed length end to end, the first opening at start_date. A cron string, written inside a CronDataIntervalTimetable, pins every window boundary to a time on the clock. A preset such as @daily is a cron string with a short name and goes in the same place. None means no schedule at all: the DAG runs only when a person triggers it, and a DAG that leaves schedule out is treated as None in Airflow 3.

A cron string has 5 fields separated by spaces: minute, hour, day of month, month and day of week. A * means every value the field allows, a number means that value only, 1-5 means a range and */15 means every 15th value. Fields are checked together, so 30 2 1 * * fires only when the minute is 30, the hour is 2 and the day of the month is 1 at the same moment. The string names a set of instants on the clock, and Airflow uses each instant as a boundary between 2 windows.

Take 0 6 * * 1-5 through the week of 2 November 2026. Its instants fall at 06:00 on Monday through Friday. The window that opens at 06:00 on Monday closes at 06:00 on Tuesday, and the run named for Monday is created at that moment. The window that opens at 06:00 on Friday 6 November has no Saturday or Sunday instant to close it, so it stays open until 06:00 on Monday 9 November: 72 hours in a single window, and its run is created on Monday morning carrying the whole weekend's data.

That is the whole method for predicting a cron schedule: list the instants, pair each with the one after it, and the run for a pair is created at the later instant and named for the earlier. Windows come out equal when the instants are evenly spaced and unequal when the string skips days, and the data each run reads stretches or shrinks to match. A task in weekday_report that reads from data_interval_start to data_interval_end handles the 72 hour window without a line of special code.

Each cron preset is read exactly as the cron string beside it, so CronDataIntervalTimetable("@daily", timezone="UTC") and the same call with "0 0 * * *" produce identical windows and identical run names. @once is the exception: it names no cron string, so it is a schedule value of its own, written as schedule="@once" and never placed inside the timetable. Pick the preset when it says what you mean and the cron string when it does not; a daily report at 06:00 has no preset and needs 0 6 * * *.

A bare string such as schedule="0 6 * * *" is accepted too. How a deployment reads a bare string is a setting on that deployment, so this lesson writes the timetable out by name and leaves the other reading to a later lesson.

start_date is the first boundary the schedule may use. The first window opens there and the first run is created when that window closes, 1 full period later. Most people read start_date as the moment the DAG first runs, deploy at 09:00 with a start_date of midnight that morning, and then watch an empty grid for 15 hours wondering what they broke.

Take daily_sales, a midnight cron with start_date 4 November 2026 00:00 UTC, deployed and unpaused at 09:00 on 4 November. Its first window is 4 November 00:00 to 5 November 00:00. That window is still open at 09:00, at 15:00 and at 23:59. At 00:00 on 5 November it closes, and the first run appears, named scheduled__2026-11-04T00:00:00+00:00. Nothing was wrong during the wait; the DAG had no complete day to count.

When start_date does not land on one of the cron's instants, the first window opens at the first instant at or after it. The same start_date of midnight with 0 6 * * * gives a first window of 4 November 06:00 to 5 November 06:00, and a first run at 06:00 on 5 November. A start_date set in the future behaves the same way: 10 November with a midnight cron yields a first run at 00:00 on 11 November.

The weekly row is the one that surprises most. A start_date on Wednesday 4 November with a Sunday schedule sits 4 days before the first instant, so the first window opens on Sunday 8 November and its run arrives on Sunday 15 November, 11 days after deployment. The future row is the cleanest way to launch on a chosen day: set 10 November, unpause whenever you like before then, and the first run arrives at midnight on the 11th with no older windows to think about.

Airflow re-reads every DAG file on a timer while it runs, so a start_date of datetime.now() is a different moment at every read. The first window opens at the latest read, and before it can close the file is read again and the window opens later still. It never closes, and the DAG never runs, with nothing in the logs to say why. Fix the date to a literal, pendulum.datetime(2026, 11, 4, tz="UTC"), and the first boundary stays put from one read to the next.

Unpause a DAG later than its start_date and some windows have already closed with no run created for them. catchup decides what happens to them. With catchup=True, Airflow creates 1 run for every closed window, oldest first, the moment the DAG is unpaused, and the runs execute side by side. With catchup=False, only the most recently closed window gets a run; the older windows never run, and nothing marks them in the grid.

Take daily_refunds, which runs at 00:00 every day from a start_date of 3 November 2026, unpaused at 06:30 on 5 November. 2 windows have closed by then: 3 to 4 November and 4 to 5 November. With catch-up on, both runs are created at 06:30 and start together. With it off, Airflow creates only the 2026-11-04 run, and 3 November is never counted. From midnight on 6 November the 2 settings behave the same, 1 run as each window closes.

The animation shows the 06:30 moment from both sides. The 2 variants have the same file apart from 1 line, and that line decides whether 3 November exists in the table the DAG writes. Airflow 3 defaults catchup to False, so a DAG that says nothing about it skips to the latest window; write the setting out either way so a reader of the file knows what will happen the day it is unpaused.

The same rule applies every time a DAG is unpaused, with the gap measured from its last scheduled run instead of from start_date. Pause daily_refunds for a week of maintenance and unpause it with catchup=True, and 7 runs are created at once; with catchup=False, 1 run is created and the other 6 days are skipped for good.

The number of catch-up runs is the number of closed windows between start_date and now, and that number grows fast with a short period. Catch-up runs begin several at a time, so a DAG with a long history behind it can occupy the workers for hours while the current day's run waits in the queue behind them. Count before you unpause.

Every run so far was created by the scheduler at the moment a window closed. A person can create one too, at any moment, by pressing the trigger button on the DAG's page in the UI or from a terminal. The result is a run like any other: it has a run_id, a logical date, a data interval and 1 task instance per task, and it moves through the same states in the grid. Its run_id begins with manual__ where a scheduled run's begins with scheduled__, and the grid marks it with its own symbol so the 2 kinds are told apart at a glance.

From a terminal, the same run comes from airflow dags trigger daily_sales. The command returns as soon as the run is created, and the run appears in the grid within a few seconds.

The trigger dialog asks for a logical date and fills in the moment you opened it. Keep that value and the run's logical date is the trigger moment, 14:20 on 5 November, and its run_id is manual__2026-11-05T14:20:00+00:00. Its data interval follows a rule of its own: the most recent window that had already closed at that moment. For daily_sales with its midnight cron, that is 4 November 00:00 to 5 November 00:00, the same window the scheduled run created at midnight was handed.

Trigger by hand on the day a DAG is deployed. Section 2 showed a first run that arrives 15 hours after deployment; a manual trigger at 09:00 produces a run at once, so every task goes green or red long before the first scheduled run. Its window is the latest closed one, 3 November, which lies before start_date, so no scheduled run will ever own it. Read the run as a check that the tasks work, and expect the scheduled run at midnight to write 4 November, the DAG's first window.

A DAG with schedule=None has only manual runs. Since it has no windows to pick from, each run is handed a window of zero length, both ends at the trigger moment, so data_interval_start and data_interval_end are equal and logical_date matches them. Such a DAG suits a job someone runs on demand, a one-off load or a cleanup, where the moment of the run is the only time worth recording.