You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "maithtruong (via GitHub)" <gi...@apache.org> on 2026/03/04 04:54:41 UTC
[D] DAG not recognized. Airflow 3.1 with Docker. [airflow]
GitHub user maithtruong created a discussion: DAG not recognized. Airflow 3.1 with Docker.
Hi everyone, can someone please help me check this problem?
I used Airflow with Docker. I did proper mappings so within the container bash, airflow-scheduler and airflow-webserver can recognize the Python file within the dags folder. But it doesn't recognize the DAG itself and wouldn't show on the UI!
So it sees that dags/flight_operations.py exists but can't see the DAG in the file, trying to list the DAG just shows "No data".
[docker-compose.yml](https://github.com/maithtruong/flight-operations/blob/main/docker-compose.yml)
[dags script](https://github.com/maithtruong/flight-operations/blob/main/dags/flight_pipeline.py)
Things I've tried doing:
- using sys.path so the container recognize the scripts in the scripts folder
- creating a Dockerfile to install the necessary dependencies
- add PYTHONPATH: /opt/airflow in environment for each of the airflow services
Thank you in advance :D.
GitHub link: https://github.com/apache/airflow/discussions/62844
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] DAG not recognized. Airflow 3.1 with Docker. [airflow]
Posted by "maithtruong (via GitHub)" <gi...@apache.org>.
GitHub user maithtruong added a comment to the discussion: DAG not recognized. Airflow 3.1 with Docker.
Sorry, I made it public now. Thank you, I'll try those solutions :D.
GitHub link: https://github.com/apache/airflow/discussions/62844#discussioncomment-16004120
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] DAG not recognized. Airflow 3.1 with Docker. [airflow]
Posted by "maithtruong (via GitHub)" <gi...@apache.org>.
GitHub user maithtruong added a comment to the discussion: DAG not recognized. Airflow 3.1 with Docker.
Ok, so I tried the solution @potiuk gave me:
- add AIRFLOW__CORE__DAG_DISCOVERY_SAFE_MODE: "False" to YAML file
I also found this link:
[Standalone DAG Processor Required](https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html#standalone-dag-processor-required)
Airflow 3.0 now requires the standalone DAG processor to parse DAGs. This dedicated process improves scheduler performance, isolation, and observability. It also simplifies architecture by clearly separating DAG parsing from scheduling logic. This change may affect custom deployments that previously used embedded DAG parsing.
So I modified the YAML file as such:
- add a separate airflow-init service and initialize the db using it (previously i used the api-server to initialize it)
- add an airflow-dag-processor service
- modify the rights so the service can access the scripts folder
I still put the scripts outside of the dags/ folder because the dag processor seem to be reading from there.
The DAGs are recognized now!
GitHub link: https://github.com/apache/airflow/discussions/62844#discussioncomment-16019788
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] DAG not recognized. Airflow 3.1 with Docker. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: DAG not recognized. Airflow 3.1 with Docker.
No access to your private repo. There is https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#dag-discovery-safe-mode that you might need to enable if your Dag fole does not contain DAG or airflow , also there might .airflowignore. It's highly recommended that **all** your scripts are in Dag Bundle folder when you are importing them (then no sys.path manipulation is needed. You can follow https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/modules_management.html
GitHub link: https://github.com/apache/airflow/discussions/62844#discussioncomment-15998288
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] DAG not recognized. Airflow 3.1 with Docker. [airflow]
Posted by "maithtruong (via GitHub)" <gi...@apache.org>.
GitHub user maithtruong closed a discussion: DAG not recognized. Airflow 3.1 with Docker.
Hi everyone, can someone please help me check this problem?
I used Airflow with Docker. I did proper mappings so within the container bash, airflow-scheduler and airflow-webserver can recognize the Python file within the dags folder. But it doesn't recognize the DAG itself and wouldn't show on the UI!
So it sees that dags/flight_operations.py exists but can't see the DAG in the file, trying to list the DAG just shows "No data".
[docker-compose.yml](https://github.com/maithtruong/flight-operations/blob/main/docker-compose.yml)
[dags script](https://github.com/maithtruong/flight-operations/blob/main/dags/flight_pipeline.py)
Things I've tried doing:
- using sys.path so the container recognize the scripts in the scripts folder
- creating a Dockerfile to install the necessary dependencies
- add PYTHONPATH: /opt/airflow in environment for each of the airflow services
Thank you in advance :D.
GitHub link: https://github.com/apache/airflow/discussions/62844
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org