Tasks in a DAG File

Concepts covered: Task dependencies

One file, one graph

A DAG is defined in a Python file placed in Airflow's dags folder. Airflow reads the file and runs its top-level code. Every DAG object that exists when the file ends becomes a DAG Airflow knows about, along with the tasks and edges attached to it. Here is the nightly report as that file, with 4 tasks built from 2 operator types.

from datetime import datetime

from airflow.sdk import DAG
from airflow.providers.standard.operators.bash import BashOperator
from airflow.providers.standard.operators.python import PythonOperator


def build_report():
    print("aggregating bookings and reviews")


with DAG(
    dag_id="nightly_bookings",
    start_date=datetime(2026, 3, 1),
    schedule="@daily",
) as dag:
    get_bookings = BashOperator(
        task_id="get_bookings",
        bash_command="python export.py bookings",
    )
    get_reviews = BashOperator(
        task_id="get_reviews",
        bash_command="python export.py reviews",
    )
    report = PythonOperator(
        task_id="build_report",
        python_callable=build_report,
    )
    publish = BashOperator(
        task_id="publish_dashboard",
        bash_command="curl -X POST https://bi.internal/refresh",
    )

    [get_bookings, get_reviews] >> report >> publish

3 arguments fix the DAG's identity. dag_id names it, and the name must be unique across the deployment. start_date is the first date the DAG is allowed to run for, and schedule="@daily" asks for one run per day; the Scheduling track covers what both mean in detail. The with DAG(...) as dag: block is the registration: every operator created inside it belongs to this DAG.

Operator in, task out

An operator is a class that describes one kind of work: run a shell command, call a Python function, send a SQL statement. Creating an instance of it inside the with block, with a task_id, creates one task on the DAG. The file above creates 3 tasks from BashOperator and one from PythonOperator. get_bookings and get_reviews are 2 tasks though they share a class, because each instance has its own task_id and its own command. A task_id must be unique within its DAG. Airflow's record identifies the task by it, and so do the UI and the logs, so the Python variable name report appears nowhere outside this file while build_report appears everywhere.

The file describes the work and does not do it

Airflow reads this file to learn the graph, and it reads it again on a timer to pick up changes. Reading means running the top-level Python: the imports, the function definition, the with block that creates 4 operator instances and 3 edges. The string "python export.py bookings" is stored as a task's argument; the export itself runs later, on a machine Airflow controls, when a run reaches that task. The same split holds for build_report: the function is defined at read time and called at run time. A line at the top of the file that queries the warehouse would run every time Airflow reads the file, dozens of times an hour, against a warehouse that has nothing to do with learning the graph.

Wiring the order

>> reads as 'then'. a >> b makes b downstream of a: b may start only after a finishes with success. << is the same edge written from the other side. A list on either end fans out or fans in: [a, b] >> c draws 2 edges into c, and a >> [b, c] draws 2 edges out of a. Chains compose, so the last line of the file draws 3 edges in one expression.

The acyclic rule is enforced at read time. Adding publish >> get_bookings to the file would make the arrows loop, so a run could never finish: publish_dashboard waits for build_report, which waits for get_bookings, which now waits for publish_dashboard. Reading the file raises an error that names the cycle, so the file fails to import: the UI lists it among the DAG import errors, with the cycle in the message, and no graph is loaded from it.

Tasks with no path between them run side by side

2 tasks may run at the same time when no path of arrows joins them in either direction. get_bookings and get_reviews have no edge between them in nightly_bookings, so when a run begins at 00:00 both start at once. build_report has 2 edges in, so it starts when the later of the 2 finishes. With the bookings export taking 20 minutes and the reviews export 10, the report starts at 00:20. Had the file said get_bookings >> get_reviews >> report, the exports would run one after the other and the report would start at 00:30. The arrows you draw decide the night's length, so draw only the ones the data requires.

About this lesson

This section is part of the DAG Anatomy: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.