Testing a DAG Locally

Concepts covered: DAG parsing

A DAG can be tried at 3 depths on a laptop, and each depth answers one question. Importing the file asks whether the DAG builds. Running one task asks whether that task's code works. Running the whole DAG asks whether the tasks work together in order. Each costs more than the one before, and knowing which question you are asking stops you paying for a 40-minute full run to find a typo that the import would have caught in a second.

Depth 1: import the file

python dags/orders_health.py imports the module the way the processor does. Zero output and exit code 0 mean every module-level statement ran without raising. That alone does not prove a DAG exists: an uncalled @dag function imports cleanly and builds nothing, as cause 3 showed. airflow dags list naming orders_health, or the depth 3 run below, confirms the DAG was built. A traceback is the same one the processor would record, shown now rather than after the next sweep. Task bodies stay unexecuted during an import, so a passing import says nothing about whether check_orders works, and a slow import on your laptop predicts a slow parse on the deployment.

Depth 2: run one task

airflow tasks test orders_health check_orders 2026-11-03 runs that single task in your terminal. It ignores the task's dependencies, so upstream tasks do not run first. It prints the task log to the screen. And it records no state to the metadata database, so nothing appears in any grid afterwards. That makes it the command to iterate on one function: edit, run, read the log, repeat, on a 5-second loop. Its blind spot follows from the first point: anything the task expects to receive from an upstream task is absent, because upstream never ran.

Depth 3: run the whole DAG

airflow dags test orders_health 2026-11-03 builds the DAG, creates 1 run stamped with that date, and executes every task in dependency order inside the one process. No scheduler or worker takes part. Task logs stream to the terminal. This command does write to the metadata database: the run and its task states are saved, and the grid shows the run afterwards. A task that fails stops its downstream tasks from running, and the command exits non-zero, so a shell script or a CI job can act on the exit code.

The date 2026-11-03 is the date the run is stamped with. What that date means for the data a run reads belongs to the scheduling lessons. When testing, reuse one date so that 2 runs can be compared.

--mark-success-pattern takes a regular expression, and any task whose id matches is marked success without running, so airflow dags test orders_health 2026-11-03 --mark-success-pattern "load_.*" skips every load task while you check what comes after them. A 20-minute wait becomes 20 seconds, and the tasks you care about still run in their real order.

from airflow.sdk import DAG, task

with DAG(dag_id="orders_health", schedule=None) as dag:
    @task
    def check_orders():
        print("ok")

    check_orders()

if __name__ == "__main__":
    dag.test()

The processor imports the file as a module, so the __main__ block never runs there and costs nothing. Once the file carries that block, running it directly becomes depth 3 with your debugger attached, and python -c "import orders_health" from inside the dags folder gives you depth 1 again.

What a green run proves

A full local run that ends green proves a specific list of things. It proves the file imports with the packages on this machine, every task's code runs to completion against the systems this machine can reach, and the dependency wiring produces an order in which that happens. All of that holds only for the machine the run happened on.

A green dags test proves

• The DAG builds with this machine's packages
• Each task completes against systems reachable from here
• The dependency order works end to end
• The task logs say what you meant them to

It stays silent on

• What the deployment's processor and workers have installed
• How the tasks behave when several run at the same time
• Anything about when a scheduler would start a run
• Whether a worker can reach the same systems

Do

• Start with the import, then one task, then the whole DAG
• Use --mark-success-pattern to skip the slow tasks you are not debugging
• Give dags test an explicit date so 2 runs are comparable
• Keep the __main__ block, since the processor never runs it

Avoid

• Treat tasks test output as a sign the DAG wires correctly
• Run dags test against the shared metadata database
• Read a green local run as proof the workers have the same packages
• Debug a parse error with a full run when the import shows it in a second

About this lesson

This section is part of the DAG Anatomy: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.