You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/21 15:03:31 UTC
[GitHub] [airflow] scrawfor opened a new issue #17138: NON_FAILED trigger rule does not work as expected.
scrawfor opened a new issue #17138:
URL: https://github.com/apache/airflow/issues/17138
**Apache Airflow version**: 2.1.2
**Environment**:
- **Cloud provider or hardware configuration**: Docker Image: apache/airflow:2.1.2-python3.8
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**:
Task with trigger_rule = "none_failed" not executed. Tasks has three upstream tasks, two of which may be skipped at certain points. Task is immediately skipped despite trigger_rule.
Same result with none_failed_or_skipped
**What you expected to happen**:
Task should wait and see that one upstream task succeeded and that it can continue.
**Anything else we need to know**:
Seems to occur every time.
```python
finalize_load = PostgresOperator(
task_id="finalize_load",
sql="sql/process_file.sql",
trigger_rule="none_failed_or_skipped"
)
[load_file_1, load_file_2, load_file_3] >> finalize_load
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] scrawfor closed issue #17138: NON_FAILED trigger rule does not work as expected.
Posted by GitBox <gi...@apache.org>.
scrawfor closed issue #17138:
URL: https://github.com/apache/airflow/issues/17138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17138: NON_FAILED trigger rule does not work as expected.
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17138:
URL: https://github.com/apache/airflow/issues/17138#issuecomment-884261651
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] scrawfor commented on issue #17138: NON_FAILED trigger rule does not work as expected.
Posted by GitBox <gi...@apache.org>.
scrawfor commented on issue #17138:
URL: https://github.com/apache/airflow/issues/17138#issuecomment-884386564
This was due to me not understanding the ShortCircuit Operator which marks all downstream tasks as skipped. I thought it only marked the current task as skipped.
Implemented a custom operator identical to ShortCircuitOperator except it gets direct tasks instead of all downstream.
```py
direct_downstream_tasks = context['task'].get_direct_relatives(upstream=False)
```
instead of
```py
downstream_tasks = context['task'].get_flat_relatives(upstream=False)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org