[GitHub] [airflow] utkarsharma2 commented on issue #19384: Add retries to LivyOperator polling / LivyHook

[GitBox <gi...@apache.org>]

utkarsharma2 commented on issue #19384:
URL: https://github.com/apache/airflow/issues/19384#issuecomment-1040757169


   @potiuk, we can implement retries in multiple ways, right now I have chosen option 1:
   
   1. We can use HttpHook’s run_with_advanced_retry method which has retry build into it via tenacity, and let user have complete control of how the retry work via choosing options by passing _retry_args_ parameter, probably good for advance users.
   2. Using same code as above option but providing default values for _retry_args_, which make sense of all users and if power user wants to overwrite, they can.
   3. We can use tenacity’s _Retrying_ method to modify call to hook.get_batch_state() at [line](https://github.com/apache/airflow/pull/21550/files#diff-b8586f3007fabbc894632e59111527536b03a2f576bda13bbcec5df9d3c1338bR153) and pass retry_args to the retryer decorated function.
   Example - 
   `retryer = Retrying(retry_args)`
   `retryer(hook.get_batch_state, batch_id)`
   
   Please let me know your opinion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org