[GitHub] [airflow] Arti3DPlayer opened a new issue, #24757: SimpleHttpOperator is not catching response for long running tasks > 20 min

[GitBox <gi...@apache.org>]

Arti3DPlayer opened a new issue, #24757:
URL: https://github.com/apache/airflow/issues/24757

   ### Apache Airflow version
   
   2.3.2 (latest released)
   
   ### What happened
   
   When I use Dag with simple http operator and task takes more than 20 min, it is not catching returned response and working forever
   
   ```
   SimpleHttpOperator(
               task_id=f'simple_http',
               http_conn_id='api',
               method='POST',
               data=json.dumps({}),
               endpoint=f'/api/test/',
               headers=headers,
               log_response=True,
               response_check=lambda response: response.status_code == 200,
           )
   ```
   
   I tried to increase `killed_task_cleanup_time` and `decrease worker_concurrency`, but this not helped. Also I found 2 same issues:
   
   https://stackoverflow.com/questions/69582817/no-response-from-request-apilonger-than-30min-while-using-simplehttpoperator
   
   https://github.com/apache/airflow/discussions/21365
   
   
   ### What you think should happen instead
   
   Operator must ended with success or failure based on response.
   
   ### How to reproduce
   
   Call an API endpoint that with sleep 30 min.
   
   ### Operating System
   
   linux
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Composer
   
   ### Deployment details
   
   Google composer
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal closed issue #24757: SimpleHttpOperator is not catching response for long running tasks > 20 min

[GitBox <gi...@apache.org>]

eladkal closed issue #24757: SimpleHttpOperator is not catching response for long running tasks > 20 min
URL: https://github.com/apache/airflow/issues/24757


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #24757: SimpleHttpOperator is not catching response for long running tasks > 20 min

[GitBox <gi...@apache.org>]

eladkal commented on issue #24757:
URL: https://github.com/apache/airflow/issues/24757#issuecomment-1172224248

   As already pointed out in previous discussion https://github.com/apache/airflow/discussions/21365#discussioncomment-2120746 this is unlikely to be Airflow issue. This report doesn't provide any insight into why do you think this is an Airflow bug?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org