You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "yc0 (via GitHub)" <gi...@apache.org> on 2023/10/02 09:36:09 UTC
[I] DagRun find_duplicate implementation is not reasonable [airflow]
yc0 opened a new issue, #34713:
URL: https://github.com/apache/airflow/issues/34713
### Apache Airflow version
2.7.1
### What happened
While we use trigger_run to trigger DAG, we sometimes get duplicate run_ids even if we actually give the different names of run_ids.
According to code https://github.com/apache/airflow/blob/4c1808bb8991c990ef2b7ef3554a5a4a5fb6078f/airflow/models/dagrun.py#L453
```
def find_duplicate(
#emit
return session.scalars(
select(cls).where(
cls.dag_id == dag_id,
or_(cls.run_id == run_id, cls.execution_date == execution_date),
)
).one_or_none()
```
Could we use `and_` instead of `or_` ? any comment or concern to adopt the or_ operator?
### What you think should happen instead
we should use `and_` instead of `or_`? any comment or concern to adopt the or_ operator?
### How to reproduce
use for_loop to execute trigger_run, you will face this issue
### Operating System
Debian GNU/Linux 11 (bullseye)
### Versions of Apache Airflow Providers
apache-airflow-providers-celery==3.3.3
apache-airflow-providers-cncf-kubernetes==7.5.0
apache-airflow-providers-common-sql==1.7.1
apache-airflow-providers-ftp==3.5.1
apache-airflow-providers-http==4.5.1
apache-airflow-providers-imap==3.3.1
apache-airflow-providers-sqlite==3.4.3
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DagRun find_duplicate implementation is not reasonable [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #34713:
URL: https://github.com/apache/airflow/issues/34713#issuecomment-1743226977
Currently both of `run_id` and `execution_date` should be unique.
I've started [a discussion](https://lists.apache.org/thread/p722zobyj8sv9vtc92pxg99wobr3blfc) about removing the `execution_date` unique constraint, and I'm checking out the different APIs/models that will be impacted. In summary, it won't be easy, but hopefully we'll see it soon.
I close the issue because if it's an expected behavior for the moment, and we can re-open it if you still think that there is a bug after my explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DagRun find_duplicate implementation is not reasonable [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala closed issue #34713: DagRun find_duplicate implementation is not reasonable
URL: https://github.com/apache/airflow/issues/34713
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org