dataengref errors

Home / Airflow / DAGs and scheduling

Airflow: AirflowTaskTimeout: DagBag import timeout for dag file

airflow.exceptions.AirflowTaskTimeoutDagBag import timeout for /opt/airflow/dags/sales_dag.py after 30.0s.

The DAG processor gave up importing a DAG file because the module took longer than [core] dagbag_import_timeout (30 seconds by default) to load. The DAG disappears or shows as an import error, even though its tasks may be fine.

Causes · Fixes · Source

Common causes

  1. Top-level code in the DAG file does real work at import time: database queries, API calls, Variable.get in a loop, or reading large files.How to confirm: Time the import with python -c "import time, runpy; t=time.time(); runpy.run_path('dags/my_dag.py'); print(time.time()-t)".
  2. The file generates many DAGs or tasks in loops, so parsing alone is slow.How to confirm: The import time grows with the number of generated DAGs or tasks.

How to fix it

Move expensive calls out of module level: into tasks, into Jinja templates, or behind functions that only run at execution time.

Raise the timeout only after removing avoidable work at import time.

SettingDefaultNote
core.dagbag_import_timeout30.0seconds

Where the message comes from

Message template in the source: DagBag import timeout for %s after %ss.

References

Related errors

Last checked: