Changing a Live Schedule

Concepts covered: Schedules and timetables

A deployed DAG runs hourly, and a request arrives to make it daily. You edit schedule, the DAG processor parses the file on its next pass, and the new serialized DAG replaces the old one as a new DAG version. The runs that already exist stay exactly as they are: their windows, logical dates and states came from the old timetable, and nothing rewrites them. The open question is where the new timetable puts the next run, and the answer comes from the method every timetable implements.

from datetime import datetime
from airflow.sdk import dag
from airflow.timetables.interval import CronDataIntervalTimetable


@dag(
    schedule=CronDataIntervalTimetable("0 0 * * *", timezone="UTC"),
    start_date=datetime(2026, 11, 1),
    catchup=True,
)
def metrics_sync():
    ...


metrics_sync()

The scheduler hands the new timetable last_automated_data_interval, the window of the latest run it created itself (scheduled or backfill, never manual or asset-triggered), and reads the next window from next_dagrun_info. The built-in data interval timetables align that window's end down to the previous tick of the new schedule and start the next window there. That one rule, plus the fact that 2 runs of a DAG cannot share a logical date, explains every case in the table.

The hourly-to-daily collision

Hourly to daily is the switch that loses data. The aligned next window is Nov 10 00:00 to Nov 11 00:00, whose logical date is Nov 10 00:00, and the run scheduled__2026-11-10T00:00:00+00:00 already exists: the hourly run for the first hour of the day holds that id. When the window closes at midnight, the scheduler finds the logical date taken, creates no run, writes a log line saying so, and moves the DAG's next window on to Nov 11 to Nov 12. Hours 00:00 to 10:00 were processed by hourly runs, and hours 10:00 to midnight by nobody. No dialog produces a 14-hour window for a daily timetable, so the recovery is a run of the task by hand over those hours, and the cheap path is to deploy the switch at a moment where no collision can happen.

Edits other than the cron string

How each edit moves the next window

• start_date moved later: The next window starts at the later of the aligned last end and the new start_date; the runs before it stay untouched
• catchup off to on: Nothing fills history before the first run; while paused, every missed window is now created on unpause instead of the latest one alone
• A cron string to an asset list: The existing scheduled runs stay, and the first event on the asset creates the first asset-triggered run
• An asset list to a cron string: If the DAG never had a scheduled or backfill run, the timetable starts as a new DAG would, from start_date under the catchup rule; otherwise it aligns from the last scheduled window as usual
• A data interval cron to a trigger cron: CronTriggerTimetable fires at the first tick after the last window's end, with a zero-length interval and that tick as logical date, so the logical date one tick earlier is never created
• A custom timetable replacing a cron: Its next_dagrun_info receives the cron run's window as last_automated_data_interval and must align it itself; a logical date already taken is skipped
• A timezone change on the cron: The aligned window shifts with the ticks, and the first new window overlaps the last old one by part of a day

Each of these edits produces a new DAG version, and the grid shows which version each run executed under. Runs created before the deploy keep the old version's windows and task list; a run created after the deploy for a window in the past, as the daily to hourly switch creates 10 of, runs the new code over the old hours. The DAG list's Next Run column shows the window the new timetable chose, so read it after the parse and before any run is created.

Do

• Pause, deploy, read Next Run on the DAG list, and unpause only if the window shown is the one you meant
• Switch to a coarser schedule right after the last fine window closes, so no logical date collides
• Work out the aligned next window from the last run's window on paper before you deploy
• Keep every task safe to run twice over the same window

Avoid

• Change schedule and start_date in one deploy and then guess which of the 2 moved the next run
• Expect existing runs' windows to be recomputed by the new schedule
• Trust that a skipped collision will page anyone: it is a line in the scheduler log

About this lesson

This section is part of the Scheduling: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.