You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2026/05/17 17:30:54 UTC
[PR] airflow-ctl-tests: wait for Dag run terminal state before xcom commands [airflow]
potiuk opened a new pull request, #67065:
URL: https://github.com/apache/airflow/pull/67065
The airflowctl xcom integration tests are flaky on ARM CI: `xcom add` plus
`xcom get` and `xcom list` succeed, then `xcom edit` and `xcom delete` fail
with `"The XCom with key … doesn't exist"`. Same SHA passes on some ARM runs
and fails on others.
**Root cause** is a race between the test's `xcom add` and execution of
`runme_0`. When the task transitions to RUNNING, the execution API queries
every XCom key currently stored for that task instance and tells the worker
to clear them via `xcom_keys_to_clear` (see
[`airflow-core/.../execution_api/routes/task_instances.py:264-275`](https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py#L264-L275)
and
[`task-sdk/.../execution_time/task_runner.py:1262-1270`](https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/execution_time/task_runner.py#L1262-L1270)).
If `xcom add` lands before the task starts running — which only takes a few
seconds for the bash echo task in `example_bash_operator` — the user-added
XCom is wiped right before `xcom edit` / `xcom delete` fires.
**Fix:** wait for the targeted Dag run to reach a terminal state
(`success`/`failed`) before each `xcom` command. Once the task is terminal it
won't run again, so the clear logic doesn't fire and the user-added XCom
survives the rest of the xcom commands.
Observed failing CI run: https://github.com/apache/airflow/actions/runs/25981599655/job/76374215829
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Opus 4.7 (1M context)
Generated-by: Claude Opus 4.7 (1M context) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow-ctl-tests: wait for Dag run terminal state before xcom commands [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #67065:
URL: https://github.com/apache/airflow/pull/67065
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] airflow-ctl-tests: wait for Dag run terminal state before xcom commands [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #67065:
URL: https://github.com/apache/airflow/pull/67065#issuecomment-4472300241
I'd love to get this one merged — and would love it in 3.2.2 if it's not too late. cc @vatsrahul1001 (3.2.2 RM)
---
Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org