Logs, Tries, and Attempts

Concepts covered: Task instance states

Every try writes its own log file, and nothing overwrites it. The worker opens the file as the cell turns running, writes a header naming the task instance and the try number, streams every line the task's code logs or prints, and closes with the state it reported. A cell that reached success on try 3 therefore has 3 files, and the 2 earlier ones hold 2 tracebacks that the green cell never mentions.

The default path under the log folder spells out the 4 names from the first section: dag_id=nightly_orders/run_id=scheduled__2026-03-09T00:00:00+00:00/task_id=load/attempt=2.log. The attempt segment is the try number, so the files of one cell sit side by side in one folder, and a listing of that folder is a history of the cell.

Which attempt to open

The task instance panel's Logs tab opens on the latest attempt and offers the others in a selector. The latest one tells you how the story ended: the final traceback for a failed cell, or the normal output for a cell that recovered. The first one tells you how it started, with the original error and its timestamp, which matters when a later try failed for a different reason, such as a lock left behind by try 1. Read the last, then the first, then compare.

• Open the latest attempt and read the last 30 lines: the state line and the traceback above it.
• Open attempt 1 and find the first line with a traceback; that is the original cause.
• Same error on every try: the fault is in the code or the data, and retries bought nothing.
• Different error on later tries: try 1 is the cause and the rest are its side effects.
• A success on the last try after failures: read attempt 1 anyway and decide if it will recur.

Take a load cell that is red after 3 tries, with retries=2 and a 5 minute delay. attempt=3.log ends on a TimeoutError raised while waiting for a lock on the orders table, and read alone it points at a slow warehouse. attempt=1.log tells a different story: try 1 wrote half its rows, then raised ConnectionResetError at 00:08 when the connection dropped. The warehouse had not yet noticed the dead connection, so the transaction try 1 opened still held the table lock when tries 2 and 3 started, and both timed out waiting on it. The fix belongs in how load handles a dropped connection; a longer retry_delay would only have hidden the cause.

A task can tell its own log which try it is in, which turns a stack of similar files into a readable sequence. get_current_context() returns the running cell, and its try_number is the same number the grid shows.

import logging
from datetime import timedelta

from airflow.sdk import get_current_context, task

log = logging.getLogger(__name__)


@task(retries=2, retry_delay=timedelta(minutes=5))
def load():
    ti = get_current_context()["ti"]
    log.info("starting try %s", ti.try_number)
    write_rows()

About this lesson

This section is part of the Task Lifecycle: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.