Airflow Task Stuck in Queued

A pool is the first suspect. Tasks without a pool use default_pool, which starts with 128 slots, and each task takes 1 slot by default. A pool you created is usually much smaller. Say warehouse_loads has 4 slots and a DAG fans out 30 tasks into it: 4 run and 26 wait. The pool is working as configured, but the board looks stuck. The pool_slots parameter makes it worse, since in the documentation's example a task with pool_slots=2 in a 2 slot pool blocks 2 other 1 slot tasks until it finishes.

Next come the caps. core.max_active_tasks_per_dag defaults to 16, so a run that fans out 40 tasks has 16 in flight and 24 waiting, unless the DAG sets its own max_active_tasks. A per task max_active_tis_per_dag limits that one task across runs. core.parallelism defaults to 32 and caps what runs at once, so a busy deployment can hold tasks in scheduled even when each DAG is under its own cap.

Priority decides who goes first, not whether anyone does. The scheduler documentation says priority "will only come into effect when there are more scheduled tasks waiting than the queue slots". When 1 DAG with a large fan out starves another, give the fan out its own pool instead of raising priorities. The scheduler applies all of these limits in each loop, evaluating 16 task instances per query by default.

With the Celery executor, the first check is that a worker is running at all: airflow celery worker starts one, and it must reach the broker. The second is the queue name. Any task can carry a queue, and a worker started with -q spark,quark picks up only tasks wired to those queues. A task sent to a queue no worker listens to is queued forever, which is the most common mistake after someone adds a queue= argument to an operator.

Capacity comes third. A Celery worker runs up to worker_concurrency tasks at once, and the provider documentation advises making sure the worker has enough resources for that many. When every worker is full, new tasks wait in queued while running ones finish. Also keep the broker's visibility_timeout above the ETA of your longest task, which the same documentation asks for, and do not clear Redis broker keys while workers run, since that can lose pending work.

With the Kubernetes executor, the equivalent question is whether the cluster can start the task's pod. The provider documentation offers airflow kubernetes generate-dag-yaml, which dumps the pods the executor would launch so you can inspect them, and its pod watcher marks a task failed when a worker pod crashes. When the scheduler process itself dies, another scheduler adopts the running or queued tasks it launched.

When a task passes scheduler.task_queued_timeout, the scheduler retries it or sets it to failed. The check is periodic, so the worst case is the timeout plus 1 interval: 600 + 120 = 720 seconds, or 12 minutes, with the defaults.

The Airflow FAQ names this as 1 of 2 reasons a task fails with no logs in the UI: the task never reached a worker, so nothing wrote a log. Setting retries on each task reduces the chance that either problem breaks a workflow. Raising the timeout hides a lack of capacity and lowering it fails tasks that only needed a slot to free up, so treat it as a safety net and fix the cause.

If the task never reaches scheduled, the problem is upstream of the executor. The FAQ's checklist covers it: the scheduler must parse the file and find the DAG object (airflow dags list shows what it found), a time based DAG waits until the first schedule interval after start_date has passed, and the scheduler "never goes back in time" to create runs. Upstream tasks must be in success, and with depends_on_past the previous instance must have succeeded or been skipped.

Run limits cause a quieter version. max_active_runs defaults to 16 through core.max_active_runs_per_dag, and runs beyond it wait, so a slow run can hold the next ones back. Clearing task instances sets the DAG run back to running, which can be what starts a task you thought was dead. The Airflow schedule interval guide explains why a first run waits a whole interval, and the Airflow DAG guide covers how runs and task instances relate.

Interviewers ask this as an operations question: a pipeline stopped moving, and what do you check. A strong answer separates the 2 states, names the gate behind each and says what you would change so it does not recur, such as a dedicated pool for a wide fan out or alerting on tasks queued longer than expected. The Airflow interview questions cover this ground, and the data pipeline interview questions and data pipeline practice problems cover designing pipelines whose concurrency limits you can defend. The system design round guide shows where it fits in a loop.