Clearing Tasks to Rerun

Concepts covered: Task instance states

Clearing a cell erases its state. The run goes back to queued, and the scheduler sees a none cell in a run whose other cells are final, so it does what it does with any none cell: checks the upstream condition, writes scheduled, and hands the task to the worker pool. The old try is never replayed. The task's code runs again from the start, as a new try with the next try number. Clearing works on any final cell, failed, upstream_failed, skipped and success alike, which is why it is the one tool you reach for both after a fix and when a green cell wrote the wrong thing.

The 5 boxes in the clear dialog

In the grid, a click on a cell opens its panel, and the Clear Task Instance button opens a dialog. The cell you clicked is always cleared. The boxes widen the selection.

• Past: Also clears this task's cell in every run before this one.
• Future: Also clears this task's cell in every run after this one.
• Upstream: Also clears every task this one depends on, so they run again first.
• Downstream: Also clears every task that depends on this one. Ticked by default.
• Only Failed: Of the cells selected, clears only those in failed or upstream_failed, leaving success cells as they are.

Downstream is the box that decides whether the run recovers. Take nightly_orders on a night when transform raised with no retries: transform is failed and load is upstream_failed. Clear transform alone and it reruns and ends success, and then nothing else happens, because load is still a final cell and the scheduler does not revisit final cells. The run ends failed a second time. Clear transform with Downstream ticked and load goes back to none in the same action, waits for transform to end success, and runs. Switch the setting under the night below to watch the 2 outcomes.

import datetime as dt
from airflow.sdk import DAG, task
from airflow.timetables.interval import CronDataIntervalTimetable

with DAG(
    dag_id="nightly_orders",
    start_date=dt.datetime(2026, 3, 1),
    schedule=CronDataIntervalTimetable("@daily", timezone="UTC"),
    catchup=False,
):
    @task
    def extract() -> None: ...
    @task(retries=0)
    def transform() -> None: ...
    @task
    def load() -> None: ...
    extract() >> transform() >> load()

The same action from a terminal: airflow tasks clear nightly_orders -t transform -s 2026-03-09 -e 2026-03-09 --downstream --yes picks the task by a regular expression on its id, the runs by a range of logical dates, and --downstream ticks the box. The --only-failed flag is the Only Failed box, and leaving out --yes prints the list of cells about to be cleared and asks first, which is the form to use on a range wider than 1 day.

Past and Future reach along a task's row, across runs, where Upstream and Downstream reach down one run's column. Clear transform in the 2026-03-09 run with Future ticked, and its cells in the 2026-03-10 and 2026-03-11 runs are cleared too, if those runs exist, and each one reruns against its own window. That is the box for a code bug that wrote bad output on several nights in a row. On a one-night failure such as a dropped connection, it reruns later cells that were already right, along with everything Downstream adds below each of them.

Clearing a green cell

Clearing a success cell is the same action with a different consequence. The scheduler reruns the task, and with Downstream ticked, everything after it. A load that appends the window's rows to a table appends them a second time, and a transform that writes a file by date overwrites it with the same content. Before clearing a green cell, read what the task writes and ask whether running it twice leaves the data the same as running it once. A later lesson is about writing tasks so the answer is always yes.

Try numbers keep counting across a clear. A load that raised on tries 1, 2 and 3, was cleared, and raised again shows try 4 in its cell, and all 4 logs stay where they were written. The next section is about those logs.

About this lesson

This section is part of the Task Lifecycle: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.