You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "steveahnahn (via GitHub)" <gi...@apache.org> on 2025/12/15 19:14:45 UTC
[PR] run id no longer collides in manual runs [airflow]
steveahnahn opened a new pull request, #59477:
URL: https://github.com/apache/airflow/pull/59477
## Problem
Manual triggers of scheduled Dags could generate the same run_id repeatedly (often aligned to the schedule boundary). Triggering the same Dag twice would then fail with a 409 Conflict due to a duplicate run_id
## Solution
When a manual trigger provides a logical_date, Airflow still infers the correct data_interval for the run, but it no longer rewrites run_after to the scheduled data_interval.end. This keeps manual trigger run_ids based on the actual trigger time making them unique across repeated manual triggers.
- Added a regression test that creates a scheduled Dag, triggers it twice, and asserts the returned dag_run_id values are different:
Manual testing in breeze triggering a daily scheduled dag via manual trigger runids:
scheduled__2025-12-15T00:00:00+00:00
manual__2025-12-15T04:16:39.584387+00:00
manual__2025-12-15T04:16:50.627593+00:00
manual__2025-12-15T04:18:07.769612+00:00
## Related
Fixes #59342
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #59477:
URL: https://github.com/apache/airflow/pull/59477#discussion_r2675675336
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py:
##########
@@ -126,10 +126,7 @@ def validate_context(self, dag: SerializedDAG) -> dict:
end=timezone.coerce_datetime(self.data_interval_end),
)
else:
- data_interval = dag.timetable.infer_manual_data_interval(
- run_after=coerced_logical_date or timezone.coerce_datetime(run_after)
- )
- run_after = data_interval.end
+ data_interval = dag.timetable.infer_manual_data_interval(run_after=coerced_logical_date)
Review Comment:
I meant why did you remove that:
```
run_after = data_interval.end
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59477:
URL: https://github.com/apache/airflow/pull/59477#issuecomment-3745393972
### Backport successfully created: v3-1-test
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-1-test</td>
<td><a href="https://github.com/apache/airflow/pull/60468"><img src="https://img.shields.io/badge/PR-60468-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #59477:
URL: https://github.com/apache/airflow/pull/59477#discussion_r2631654573
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py:
##########
@@ -126,10 +126,7 @@ def validate_context(self, dag: SerializedDAG) -> dict:
end=timezone.coerce_datetime(self.data_interval_end),
)
else:
- data_interval = dag.timetable.infer_manual_data_interval(
- run_after=coerced_logical_date or timezone.coerce_datetime(run_after)
- )
- run_after = data_interval.end
+ data_interval = dag.timetable.infer_manual_data_interval(run_after=coerced_logical_date)
Review Comment:
Can you explain why you removed the `data_interval` being inferred from the `run_after` date if the `coerced_logical_date` is None?
This piece doesn't seem directly related to the fix. (just removing the `run_after = data_interval.end` should be enough)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #59477:
URL: https://github.com/apache/airflow/pull/59477#discussion_r2675675336
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py:
##########
@@ -126,10 +126,7 @@ def validate_context(self, dag: SerializedDAG) -> dict:
end=timezone.coerce_datetime(self.data_interval_end),
)
else:
- data_interval = dag.timetable.infer_manual_data_interval(
- run_after=coerced_logical_date or timezone.coerce_datetime(run_after)
- )
- run_after = data_interval.end
+ data_interval = dag.timetable.infer_manual_data_interval(run_after=coerced_logical_date)
Review Comment:
Nevermind I got why it's necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #59477:
URL: https://github.com/apache/airflow/pull/59477#discussion_r2631654573
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py:
##########
@@ -126,10 +126,7 @@ def validate_context(self, dag: SerializedDAG) -> dict:
end=timezone.coerce_datetime(self.data_interval_end),
)
else:
- data_interval = dag.timetable.infer_manual_data_interval(
- run_after=coerced_logical_date or timezone.coerce_datetime(run_after)
- )
- run_after = data_interval.end
+ data_interval = dag.timetable.infer_manual_data_interval(run_after=coerced_logical_date)
Review Comment:
Can you explain why you removed the `data_interval` being inferred from the `run_after` date if the `coerced_logical_date` is None?
This piece doesn't seem directly related to the fix. (just removing the `run_after = data_interval.end` should be enough)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #59477:
URL: https://github.com/apache/airflow/pull/59477
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unique run_id across manually triggered Dags with schedules [airflow]
Posted by "steveahnahn (via GitHub)" <gi...@apache.org>.
steveahnahn commented on code in PR #59477:
URL: https://github.com/apache/airflow/pull/59477#discussion_r2649453537
##########
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py:
##########
@@ -126,10 +126,7 @@ def validate_context(self, dag: SerializedDAG) -> dict:
end=timezone.coerce_datetime(self.data_interval_end),
)
else:
- data_interval = dag.timetable.infer_manual_data_interval(
- run_after=coerced_logical_date or timezone.coerce_datetime(run_after)
- )
- run_after = data_interval.end
+ data_interval = dag.timetable.infer_manual_data_interval(run_after=coerced_logical_date)
Review Comment:
We are under ```if coerced_logical_date:``` conditional, therefore ```coerced_logical_date``` is guaranteed to exist so I removed the remaining dead code from the or statement and cleaned up the variable assignment. The actual fix is removing run_after = data_interval.end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org