What the Logical Date Names

Concepts covered: Data intervals

Take a DAG whose schedule is CronDataIntervalTimetable("0 6 * * *", timezone="UTC"), with a start_date of 1 November 2026 and catchup on. At 06:00 UTC on 4 November the scheduler creates a run for the window from 3 November 06:00 to 4 November 06:00. That run's logical_date is 2026-11-03T06:00:00+00:00, the instant the window opened, and its run_id is scheduled__2026-11-03T06:00:00+00:00. The clock reads the 4th; the run is named for the 3rd.

The logical date names which slice of the schedule a run stands for. What is unique per DAG is the run id, and a scheduled run's id is built from its logical date, so the scheduler can tell from the id alone whether that slice already has a scheduled run and never creates a second. A fourth field, run_after, records the earliest instant the run was allowed to start: on this timetable, the moment the window closed. The table below is the run list after 3 scheduled runs, and in every row the logical date and the window's start agree while run_after sits 24 hours later.

Where the logical date shows up

• In the run id, after the scheduled__ prefix, written in UTC with its offset.
• As the column header of each run in the grid, so the grid reads one period behind the clock on this timetable.
• In the dialog that picks a run to clear or inspect, which lists runs by it.

Name and window diverge

What the logical date equals depends on the timetable. A data-interval timetable makes it the window's start. A trigger timetable, which section 1 builds, makes it the instant the cron fired, and the window has zero length. A run triggered by hand in 3.3 may have no logical date at all, and a run created without one carries data_interval_start and data_interval_end of None as well. A task that derives its window from logical_date has assumed a timetable. A task that reads the 2 window fields has been told the window by whichever timetable is in force, and keeps working when someone changes it.

from airflow.sdk import task


@task
def load_orders(data_interval_start=None, data_interval_end=None):
    start = data_interval_start.strftime("%Y-%m-%d %H:%M:%S")
    end = data_interval_end.strftime("%Y-%m-%d %H:%M:%S")
    return {"start": start, "end": end}

The run named 3 November returns a start of 2026-11-03 06:00:00 and an end of 2026-11-04 06:00:00. The filter they drive is half-open: rows at or after the start and strictly before the end. A row stamped exactly 06:00:00 on the 4th therefore belongs to the next run and to no other, and 2 neighbouring runs never count the same row twice or drop one between them.

SELECT
    count(*) AS orders
FROM orders
WHERE created_at >= TIMESTAMP '2026-11-03 06:00:00'
    AND created_at < TIMESTAMP '2026-11-04 06:00:00'

Where ds goes wrong

ds is the logical date's calendar day, 2026-11-03 for this run. Read as the day this run is for, it names a day whose first 6 hours this run never touches, and it leaves out the 6 hours of 4 November the run does read. ds matches the window only when the window is a calendar day in the timetable's zone. On a trigger timetable it names the firing day while the window is empty. Reading data_interval_start and data_interval_end gives the same answer under every timetable, so a task written that way survives a schedule change with no edit, and a backfilled run of it reads the old window rather than today's.

import pendulum
from airflow.sdk import dag, task
from airflow.timetables import (
    interval as tt)

@dag(
    schedule=tt.CronDataIntervalTimetable(
        "0 6 * * *", timezone="UTC"),
    start_date=pendulum.datetime(
        2026, 11, 1, tz="UTC"),
    catchup=True,
)
def orders_daily():
    @task
    def load_orders(data_interval_start=None):
        pass
    load_orders()
orders_daily()

About this lesson

This section is part of the Scheduling: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.