Airflow DAG Not Showing in UI

A file that raises while it's imported never registers its DAGs. The UI reports the failure, and airflow dags list-import-errors prints the same list from the command line. The traceback shown is trimmed to 2 entries by default, so the bottom of the stack is the line that matters. Set core.dagbag_import_error_traceback_depth higher when your own helper modules hide the real cause.

Most import errors are ordinary Python errors, usually a missing package in the environment of the Dag processor, or a syntax error or typo in an import. The environment is the usual surprise. The processor imports your file with its own Python environment, so a package installed on the workers only is invisible to it. To reproduce the error, run the file with the same interpreter and environment the processor uses.

Slow imports look like missing DAGs too. core.dagbag_import_timeout limits a file import to 30 seconds, and dag_processor.dag_file_processor_timeout limits a whole parse to 50 seconds. The FAQ warns that when parsing takes longer than the import timeout, files may not be processed completely. Keep network calls, database queries and heavy imports out of top-level code.

With the default core.dag_discovery_safe_mode of True, Airflow reads only files that contain the strings airflow and dag, ignoring case. The FAQ says it "ignores files not containing airflow and DAG in order to prevent the DagBag parsing" of unrelated scripts. A file that builds its DAGs through a wrapper from another module may contain neither string. Add a comment with both, or set the option to False. The processor reads it, so set it on the dag-processor and on anything that runs airflow dags reserialize.

Airflow takes only DAG objects at the top level of a module. The Dags documentation shows a DAG created inside a function as not loaded. If a factory function builds your DAGs, assign each result to a module level name, or call the factory at module level so the objects exist where the loader looks.

A .airflowignore file lists patterns the loader skips, and its scope is its directory plus all subfolders. When a directory name matches a pattern, that directory and everything under it is not scanned. A DAG that lives in a folder you added to the ignore list long ago is a quiet cause.

A DAG not found message means the id you asked for is not in Airflow's metadata. The causes are the ones above, so start with airflow dags list, which lists all the DAGs, and airflow dags list-import-errors. If the id appears in neither, the file hasn't been parsed. Check the id for a typo too, since the id is the dag_id argument, not the file name.

A second cause is asking the wrong place. A command run in your shell uses your shell's configuration, which can name a different DAGs folder or database than the deployment you look at in the browser. Compare dags_folder and the metadata connection on both, and keep the processor's configuration the reference.

A DAG that was there and is gone was deactivated. The documentation says deactivation happens by removing files from the DAGs folder and cannot be done from the UI or API; the DAG's metadata and history are kept. The processor deactivates DAGs missing from a file 50 seconds after the file is re-parsed by default, so a file that now fails or no longer defines the DAG removes it. A paused DAG stays listed, and you can trigger it by hand.

Interviewers ask this as an operations question: a deploy went out and the DAG is not in the UI, so what do you check. A strong answer walks the path in order: the processor is running, the file is in the folder it reads, the import succeeds in its environment, and the 300 second refresh has passed. Then it says how you would stop it recurring, such as running an import check for DAG files in CI. The Airflow interview questions cover the scheduler side of this ground, and the data pipeline interview questions and data pipeline practice problems cover designing pipelines you can operate. The system design round guide shows where it fits in a loop.

When the DAG shows but its tasks do not run, the problem has moved on to the scheduler: the task stuck in queued guide covers that. The Airflow DAG guide explains what a DAG is and how runs relate to it.