You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2019/04/05 23:46:29 UTC
[GitHub] [airflow] mik-laj commented on issue #5043: [AIRFLOW-4236] Add
num_retries to MySqlToGoogleCloudStorageOperator
mik-laj commented on issue #5043: [AIRFLOW-4236] Add num_retries to MySqlToGoogleCloudStorageOperator
URL: https://github.com/apache/airflow/pull/5043#issuecomment-480453668
Other operators have a fixed number of retry attempts.
https://github.com/apache/airflow/blob/e27950a75ce287c094e550fba07d1c8de5dc4143/airflow/contrib/hooks/gcp_transfer_hook.py#L92
https://github.com/apache/airflow/blob/383ec374ea148d04af94496dc9e3283263d67d5f/airflow/contrib/hooks/gcp_function_hook.py#L29
https://github.com/apache/airflow/blob/993779f93df172f5f774f1ea2634c55e9c1212c4/airflow/contrib/hooks/gcp_compute_hook.py#L28
https://github.com/apache/airflow/blob/49ade3834a46d9a12c0339a017d1807bf021c5e3/airflow/contrib/hooks/gcp_sql_hook.py#L54
https://github.com/apache/airflow/blob/85ea7e8575bb0f9a2fac989f26970809b36640a4/airflow/contrib/hooks/gcp_dataflow_hook.py#L51
I do not know if it's a good idea to give the user the option to configure this parameter in this way. Too many parameters will make it difficult to use the operator. In my opinion, this value should be configured in the connection settings instead of the specific use, but fixed value is good enough in this case.
I working on similar mechanism for Google Cloud Vision: https://github.com/PolideaInternal/airflow/pull/85/files
I want to support retrying in reaction to exceeding the temporary quota. I would like the use of all GCP operators to be similar, so it is important for me to elaborate one variant.
CC: @potiuk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services