You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "alvaromaroto (via GitHub)" <gi...@apache.org> on 2023/09/19 10:28:22 UTC

[GitHub] [airflow] alvaromaroto opened a new issue, #34471: SIGTERM not catched on callbacks

alvaromaroto opened a new issue, #34471:
URL: https://github.com/apache/airflow/issues/34471

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   **Context**
   
   I'm trying to execute a code when I mark a task as failed. For instance, I want to launch a function that cancel a job in Tableau Server when I mark a task as `failed`.
   
   <img width="579" alt="image" src="https://github.com/apache/airflow/assets/44638467/c64bfe04-b6fc-460c-b579-6505f87cf120">
   
   I see that when a task fails due to another error, a `on_failure_callback` or `on_retry_callback` function are invoked. However, I've checked that these functions are not invoked when you manually cancel the task. I only see that the `SIGTERM` signal is sent to the task but no code is executed.
   
   ### What you think should happen instead
   
   <img width="1420" alt="image" src="https://github.com/apache/airflow/assets/44638467/0059db19-31dc-4e6b-be3a-157ba2535522">
   
   I think that the `on_failure_callback` is not invoked when a SIGTERM is sent to the task.
   
   
   ### How to reproduce
   
   1 - Create a simple DAG with a `on_failure_callback` with a dummy print. For instance:
   
   ```python
   def sleeping():
       from time import sleep
       print("Sleeping until fail...")
       sleep(100000)
   
   def test_callback():
     print("Calling when task is failing")
   
   dag = DAG('hello_world', schedule_interval='0 12 * * *',
             start_date=datetime(2023, 7, 20), on_failure_callback=test_callback)
   
   hello_operator = PythonOperator(task_id='hello_task', python_callable=sleeping, dag=dag, on_failure=test_callback)
   ```
   
   2- Once is running, mark the task as failed and test that the callback is not invoked
   
   ### Operating System
   
   MacOS
   
   ### Versions of Apache Airflow Providers
   
   2.6.0
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] SIGTERM not catched on callbacks [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis closed issue #34471: SIGTERM not catched on callbacks
URL: https://github.com/apache/airflow/issues/34471


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ephraimbuddy commented on issue #34471: SIGTERM not catched on callbacks

Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on issue #34471:
URL: https://github.com/apache/airflow/issues/34471#issuecomment-1730791116

   It’s currently not supported see https://github.com/apache/airflow/blob/08b45ba09fe713c0c3fb2dc94b9926fab76e4e65/airflow/models/taskinstance.py#L1564


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] hussein-awala commented on issue #34471: SIGTERM not catched on callbacks

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #34471:
URL: https://github.com/apache/airflow/issues/34471#issuecomment-1732683832

   > Do you think that it is going to be supported in further versions? thanks man
   
   @alvaromaroto IMHO it should not trigger `on_failure_callback` because in case of dag run timeout or a manual clear, we cannot consider that the TI was failed. But for the clear case we can add a new callback `on_clear_callback` which makes more sense, and it will be up to the user whether he want to call the `on_failure_callback` method or another method.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] alvaromaroto commented on issue #34471: SIGTERM not catched on callbacks

Posted by "alvaromaroto (via GitHub)" <gi...@apache.org>.
alvaromaroto commented on issue #34471:
URL: https://github.com/apache/airflow/issues/34471#issuecomment-1731128507

   Thanks @ephraimbuddy Do you think that it is going to be supported in further versions? thanks man


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org