You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "luc-pimentel (via GitHub)" <gi...@apache.org> on 2026/10/03 21:30:36 UTC
[I] Short-circuit in a mapped task group lets later all_done and none_failed tasks run [airflow]
luc-pimentel opened a new issue, #74175:
URL: https://github.com/apache/airflow/issues/74175
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
main (as of 2026-10-03); also reproduced on 3.1.8
### What happened and how to reproduce it?
Inside a mapped task group, a short-circuit task that returns False skips only the task right after it. Later tasks of the same map index still run if their trigger rule accepts a skipped upstream (`all_done`, `none_failed`), even with the default `ignore_downstream_trigger_rules=True`.
```python
import pendulum
from airflow.sdk import dag, task, task_group
@dag(schedule=None, start_date=pendulum.datetime(2026, 1, 1, tz="UTC"), catchup=False)
def short_circuit_mapped_group():
@task_group
def group(value):
@task.short_circuit # ignore_downstream_trigger_rules=True (default)
def gate(v):
return v
@task
def a():
pass
@task(trigger_rule="all_done") # "none_failed" behaves the same
def b():
pass
gate(value) >> a() >> b()
group.expand(value=[True, False])
short_circuit_mapped_group()
```
Steps: run `airflow standalone` (SQLite, LocalExecutor), trigger the Dag and check the task instances of map index 1. `airflow dags test short_circuit_mapped_group` gives the same states.
Map index 1 (`gate` returns False):
| Build | `group.a[1]` | `group.b[1]`, `all_done` | `group.b[1]`, `none_failed` |
| --- | --- | --- | --- |
| 3.1.8 | skipped | success | success |
| main | skipped | success | success |
Map index 0 runs everything, as expected. Controls on the same builds:
- The same chain outside a mapped task group: `a` and `b` are skipped.
- `ignore_downstream_trigger_rules=False` in the mapped group: `b[1]` runs, the same as with `True`.
- `b` with the default `all_success`: `b[1]` is skipped, because the skip propagates through `a`.
### What you think should happen instead?
`group.b[1]` should be skipped. The ShortCircuitOperator docs say that with `ignore_downstream_trigger_rules=True`, "all downstream tasks are skipped without considering the `trigger_rule` defined for tasks". That holds outside a mapped task group, but inside one the flag currently has no effect.
The XCom that `gate` writes for map index 1 lists both `group.a` and `group.b`. For a mapped task instance, `SkipMixin._set_state_to_skipped` in `task-sdk/src/airflow/sdk/bases/skipmixin.py` does not skip anything itself and leaves it to `NotPreviouslySkippedDep`, which only reads the decision of direct upstream tasks, so only `group.a` picks it up.
### Operating System
macOS 26.2
### Deployment
Virtualenv installation
### Apache Airflow Provider(s)
standard
### Versions of Apache Airflow Providers
apache-airflow-providers-standard 1.12.0 (with 3.1.8); main installed from source.
### Official Helm Chart version
Not Applicable
### Kubernetes Version
Not Applicable
### Helm Chart configuration
Not Applicable
### Docker Image customizations
Not Applicable
### Anything else?
It happens every time. The main build I tested already includes #73959 and #73960, the two fixes from #55225; this case is not covered by either. On 3.2.0 to 3.3.2, nothing inside a mapped task group is skipped at all, not even `group.a[1]`, which #73959 fixed on main. #73960 covers a different case: an unmapped short-circuit with an expanded mapped downstream task.
---
Drafted-by: Claude Code (Opus 5.5); reviewed by @luc-pimentel before posting
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Short-circuit in a mapped task group lets later all_done and none_failed tasks run [airflow]
Posted by "anzinmhd (via GitHub)" <gi...@apache.org>.
anzinmhd commented on issue #74175:
URL: https://github.com/apache/airflow/issues/74175#issuecomment-5979513164
I’ve reproduced the issue and identified the cause in `NotPreviouslySkippedDep`.
The fix updates it to consider all upstream relatives so that skip decisions from a `ShortCircuitOperator` inside a mapped task group are correctly respected by transitive downstream tasks.
I’ve also added a regression test and verified that the reported behavior is fixed.
A PR is being opened with the fix: `Fix mapped task group short circuit skipping`.
If this approach needs any changes or there’s a better way to address the issue, I’m happy to discuss and adjust the implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Short-circuit in a mapped task group lets later all_done and none_failed tasks run [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil closed issue #74175: Short-circuit in a mapped task group lets later all_done and none_failed tasks run
URL: https://github.com/apache/airflow/issues/74175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Short-circuit in a mapped task group lets later all_done and none_failed tasks run [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on issue #74175:
URL: https://github.com/apache/airflow/issues/74175#issuecomment-5977280650
> Hey [@luc-pimentel](https://github.com/luc-pimentel), I would like to work on this can you assign this to me
Currently we don't assign contributors to issues according to policy (with some exceptions).
Feel free to create a PR without assignment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Short-circuit in a mapped task group lets later all_done and none_failed tasks run [airflow]
Posted by "akshat-lakhera (via GitHub)" <gi...@apache.org>.
akshat-lakhera commented on issue #74175:
URL: https://github.com/apache/airflow/issues/74175#issuecomment-5975737216
Hey @luc-pimentel, I would like to work on this can you assign this to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org