You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "hussein-awala (via GitHub)" <gi...@apache.org> on 2023/08/20 18:24:23 UTC
[GitHub] [airflow] hussein-awala opened a new issue, #33561: The TI doesn't wait setup task when they are not a direct upstream
hussein-awala opened a new issue, #33561:
URL: https://github.com/apache/airflow/issues/33561
### Apache Airflow version
2.7.0
### What happened
When we add a setup task (let's call it S) before a sequence of tasks, and we clear the state of a not direct downstream task for the setup task (let's call it T), the state of S is correctly cleared, but the task T doesn't wait for S, they are running in parallel.
### What you think should happen instead
The setup task should be treated as an upstream for all the downstream tasks
### How to reproduce
```python
from datetime import datetime
from airflow.models import DAG
from airflow.operators.python import PythonOperator
def sleep_and_log(msg):
import time
time.sleep(5)
print(msg)
with DAG(
dag_id="test_setup_teardown",
schedule_interval=None,
start_date=datetime(2023, 8, 20),
catchup=False,
):
setup_task = PythonOperator(
task_id="setup_task",
python_callable=lambda: sleep_and_log("setup_task"),
).as_setup()
teardown_task = PythonOperator(
task_id="teardown_task",
python_callable=lambda: sleep_and_log("teardown_task"),
).as_teardown(setups=[setup_task])
t1 = PythonOperator(
task_id="t1",
python_callable=lambda: sleep_and_log("t1"),
)
t2 = PythonOperator(
task_id="t2",
python_callable=lambda: sleep_and_log("t2"),
)
setup_task >> t1 >> t2 >> teardown_task
```
Create a run and wait all the tasks to finish, then clear the state of t2, the setup task and t2 states will be cleared, and both tasks will run in parallel.
### Operating System
Debian GNU/Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The TI doesn't wait setup tasks when they are not a direct upstream [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #33561:
URL: https://github.com/apache/airflow/issues/33561#issuecomment-2301283934
This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The TI doesn't wait setup tasks when they are not a direct upstream [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #33561: The TI doesn't wait setup tasks when they are not a direct upstream
URL: https://github.com/apache/airflow/issues/33561
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The TI doesn't wait setup tasks when they are not a direct upstream [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #33561:
URL: https://github.com/apache/airflow/issues/33561#issuecomment-2365621998
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org