You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "keu (via GitHub)" <gi...@apache.org> on 2023/02/02 15:12:25 UTC

[GitHub] [airflow] keu opened a new issue, #29321: Airflow doesn't show an error when there are DAGs with the same ID

keu opened a new issue, #29321:
URL: https://github.com/apache/airflow/issues/29321

   ### Apache Airflow version
   
   2.5.1
   
   ### What happened
   
   Create two files with DAGs
   Both DAGs should have the same ID
   Airflow will constantly update DAG structure/code resulting in dozens of weir behaviours and errors:
   - wrong DAG visualization
   - wrong DAG code
   - errors about missing task from time to timebehaviors
   
   ### What you think should happen instead
   
   Airflow should show an error to the user and disable both DAGs.
   
   ### How to reproduce
   
   1. Create two files with DAGs
   2. Both DAGs should have the same ID
   
   
   ### Operating System
   
   Ubuntu 22.04
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal closed issue #29321: Airflow doesn't show an error when there are DAGs with the same ID

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #29321: Airflow doesn't show an error when there are DAGs with the same ID
URL: https://github.com/apache/airflow/issues/29321


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #29321: Airflow doesn't show an error when there are DAGs with the same ID

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #29321:
URL: https://github.com/apache/airflow/issues/29321#issuecomment-1419265597

   @michaelmicheal see https://github.com/apache/airflow/issues/19525#issuecomment-977818865 for more information
   
   Closing as duplicate of https://github.com/apache/airflow/pull/17891 @michaelmicheal if you'd like to work on it please comment on that issue so we can assign you
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29321: Airflow doesn't show an error when there are DAGs with the same ID

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #29321:
URL: https://github.com/apache/airflow/issues/29321#issuecomment-1413902029

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] michaelmicheal commented on issue #29321: Airflow doesn't show an error when there are DAGs with the same ID

Posted by "michaelmicheal (via GitHub)" <gi...@apache.org>.
michaelmicheal commented on issue #29321:
URL: https://github.com/apache/airflow/issues/29321#issuecomment-1418455335

   As I understand it, as part of DAG processing, when the scheduler sees new a definition for an existing DAG, it overwrites regardless of the path. While this is by design, my team's stakeholders face this quite a bit where user's have duplicate files, with slightly different versions of a DAG. It's not obvious, and if you rely on a GCS or S3 DAG bucket that is abstracted from users/developers, it can be hard to diagnose.
   
   One solution could be:
   If the DAG processor sees an existing DAG defined in a file under a different path, we re-process the file under the old path, and see if the DAG is still defined there. If the DAG is still defined under the old path, we add an import error.
   
   
   Feel free to assign this to me.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org