You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/10 18:42:26 UTC

[GitHub] [airflow] MatrixManAtYrService opened a new issue #19525: Duplicate dag_id should Emit an error

MatrixManAtYrService opened a new issue #19525:
URL: https://github.com/apache/airflow/issues/19525


   ### Apache Airflow version
   
   2.2.2rc1 (release candidate)
   
   ### Operating System
   
   debian (docker)
   
   ### Versions of Apache Airflow Providers
   
   n/a
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   Two dags:
   
   ```
   # one.py
   
   @task
   def a():
       print("a")
   
   @dag(schedule_interval=None, start_date=days_ago(2))
   def my_dag():
       a()
   
   dag = my_dag()
   ```
   
   ```
   # two.py
   
   @task
   def b():
       print("b")
   
   @dag(schedule_interval=None, start_date=days_ago(2))
   def my_dag():
       b()
   
   dag = my_dag()
   ```
   
   Note that they share the same dag_id: `my_dag`
   
   
   ### What happened
   
   Only one DAG appeared.  In the tree-view, I would refresh the page and see the task change between:
   - task_id: a
   - task_id: b
   
   seemingly at random.
   
   ### What you expected to happen
   
   At least, I expected a warning to appear.  It seems that we used to have this: https://github.com/apache/airflow/pull/15302
   
   I think that in the presence of a dag_id collision, we should either:
   - refuse to run that dag at all until the collision is resolved
   - resolve it deterministically
   
   ### How to reproduce
   
   Include the two dags above and observe the "DAGs" view.  Notice that there's only one DAG and it's not clear which one it is.
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr closed issue #19525: Duplicate dag_id should emit an error

Posted by GitBox <gi...@apache.org>.
uranusjr closed issue #19525:
URL: https://github.com/apache/airflow/issues/19525


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on issue #19525: Duplicate dag_id should emit an error

Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #19525:
URL: https://github.com/apache/airflow/issues/19525#issuecomment-977818865


   #15302 only handles duplicated IDs in a DAG processor, not in the entire Airflow setup. That is very difficult to do, as already discussed in depth in #17861 and #17891.
   
   Closing this as a duplicate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #19525: Duplicate dag_id should emit an error

Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #19525:
URL: https://github.com/apache/airflow/issues/19525#issuecomment-977732196


   cc @uranusjr seems like edge case of https://github.com/apache/airflow/pull/15302 ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org