You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/29 16:05:24 UTC

[GitHub] [airflow] lidalei opened a new issue #9569: HttpHook.run_with_advanced_retry method does not support Exception other than ConnectionError

lidalei opened a new issue #9569:
URL: https://github.com/apache/airflow/issues/9569


   
   **Apache Airflow version**:
   1.10.10
   
   **Environment**:
   Python 3.6.10
   
   **What happened**:
   The following code block
   ```
   hook = HttpHook(method='GET', http_conn_id='xxx')
   
   retry_args = dict(
       wait=tenacity.wait_exponential(),
       stop=tenacity.stop_after_attempt(10),
       retry=retry_if_exception_type(
           exception_types=(requests.exceptions.ConnectionError, requests.exceptions.HTTPError)
       ),
   )
   
   hook.run_with_advanced_retry(retry_args, ...)
   ```
   does not retry an HTTP request and raises an AirflowException when `check_response` is True (by default) and the server responds a 429 status code (too many requests).
   
   **What you expected to happen**:
   I expect the `retry` parameter is respected. So instead of raising `AirflowException`, the original exception should be raised.
   
   This is because `check_response` changes the Exception type from `HTTPError` to `AirflowException` [check_response](https://github.com/apache/airflow/blob/96697180d79bfc90f6964a8e99f9dd441789177c/airflow/hooks/http_hook.py#L139-L152). However, only requests.exceptions.ConnectionError is [caught](https://github.com/apache/airflow/blob/96697180d79bfc90f6964a8e99f9dd441789177c/airflow/hooks/http_hook.py#L180-L184).
   
   
   **How to reproduce it**:
   You can understand the issue by reading the source code, without actually doing something to reproduce it.
   
   **Anything else we need to know**:
   The function document of [run_with_advanced_retry](https://github.com/apache/airflow/blob/96697180d79bfc90f6964a8e99f9dd441789177c/airflow/hooks/http_hook.py#L205) is wrong. It should be `retry=retry_if_exception_type(exception_types=(requests.exceptions.ConnectionError,))`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] boring-cyborg[bot] commented on issue #9569: HttpHook.run_with_advanced_retry method does not support Exception other than ConnectionError

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #9569:
URL: https://github.com/apache/airflow/issues/9569#issuecomment-651214951


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org