You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/02 21:14:41 UTC
[GitHub] [airflow] Fahadsaadullahkhan opened a new issue #12765: Success call back function running twice
Fahadsaadullahkhan opened a new issue #12765:
URL: https://github.com/apache/airflow/issues/12765
**Apache Airflow version**: 1.10.12
**Google Cloud Composer**: composer-1.13.0-airflow-1.10.12
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.16.13-gke.404
**Environment**:
- **Cloud provider or hardware configuration**: Google cloud
- **OS** (e.g. from /etc/os-release): Container-Optimized OS from Google
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**:
So I have a call back function on success and failed tasks, so for success call back sometimes success call back function is running two times, one when task gets completed and second time even after task gets marked as success which shouldn't happen. Please see logs:
`[2020-12-02 19:00:40,306] {base_task_runner.py:113} INFO - Job 3703: Subtask send_airflow_url [2020-12-02 19:00:40,306] {base_hook.py:89} INFO - Using connection to: id: ********.default, Port: None, Schema: composer-1-13-0-airflow-1-10-12-29ebacc4, Login: root, Password: XXXXXXXX, extra: None
[2020-12-02 19:00:40,764] {logging_mixin.py:112} INFO - Response: ok
[2020-12-02 19:00:40,770] {base_task_runner.py:113} INFO - Job 3703: Subtask send_airflow_url [2020-12-02 19:00:40,764] {logging_mixin.py:112} INFO - Response: ok **(Success call back logs)**
[2020-12-02 19:00:40,785] {logging_mixin.py:112} INFO - task_count: 11, state_count: 0, skip_state_count:0
[2020-12-02 19:00:40,786] {base_task_runner.py:113} INFO - Job 3703: Subtask send_airflow_url [2020-12-02 19:00:40,785] {logging_mixin.py:112} INFO - task_count: 11, state_count: 0, skip_state_count:0
[2020-12-02 19:00:40,788] {taskinstance.py:1072} INFO - Marking task as SUCCESS.dag_id=ing1606935387856, task_id=send_airflow_url, execution_date=20201202T190005, start_date=20201202T190021, end_date=20201202T190040
[2020-12-02 19:00:40,791] {base_task_runner.py:113} INFO - Job 3703: Subtask send_airflow_url [2020-12-02 19:00:40,788] {taskinstance.py:1072} INFO - Marking task as SUCCESS.dag_id=ing1606935387856, task_id=send_airflow_url, execution_date=20201202T190005, start_date=20201202T190021, end_date=20201202T190040
[2020-12-02 19:00:42,674] {logging_mixin.py:112} INFO - Response: ok **(Success call back logs)**
[2020-12-02 19:00:42,687] {logging_mixin.py:112} INFO - task_count: 11, state_count: 1, skip_state_count:0
[2020-12-02 19:00:42,689] {helpers.py:325} INFO - Sending Signals.SIGTERM to GPID 116278
[2020-12-02 19:00:42,689] {helpers.py:291} INFO - Process psutil.Process(pid=116278, status='terminated', exitcode=0, started='19:00:22') (116278) terminated with exit code 0
`
**What you expected to happen**:
Success call back should only run once not twice.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-738113577
It's not merged yet, but yes targeting 2.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-738102644
Will likely be fixed by https://github.com/apache/airflow/pull/10917
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vikramkoka commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
vikramkoka commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-737636087
Can you please share how to reproduce this issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-737499276
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Fahadsaadullahkhan commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
Fahadsaadullahkhan commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-738095011
> Can you please share how to reproduce this issue?
Create a composer environment with composer-1.13.0-airflow-1.10.12 image then you can run a sample dag which has the success and failed call back in it. other than that I'm not sure what you need to reproduce this issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Fahadsaadullahkhan commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
Fahadsaadullahkhan commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-738106685
> Will likely be fixed by #10917
Which version will have this fix? 2.0?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #12765:
URL: https://github.com/apache/airflow/issues/12765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ohashmi-cei commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
ohashmi-cei commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-738066795
**Apache Airflow version: 1.10.12
Google Cloud Composer: composer-1.13.0-airflow-1.10.10**
Hi I am having the same issue, the success call back function is called twice on random dags. I am attaching the picture of how it marks the dags as successful twice in the logs. There are no loops in the flow of my dags, it runs as following.
run_search >> delete_cluster
<img width="1386" alt="Screen Shot 2020-12-03 at 9 55 12 AM" src="https://user-images.githubusercontent.com/57103526/101047481-df4d3680-354f-11eb-8bae-867337f45fc9.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-942201814
Closing as the issue is fixed in Airflow 2.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #12765:
URL: https://github.com/apache/airflow/issues/12765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #12765: Success call back function running twice
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #12765:
URL: https://github.com/apache/airflow/issues/12765#issuecomment-942201814
Closing as the issue is fixed in Airflow 2.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org