[jira] [Created] (AIRFLOW-4236) Operators uploading to GCS needs to handle googleapiclient.errors

["jack (JIRA)" <ji...@apache.org>]

jack created AIRFLOW-4236:
-----------------------------

             Summary: Operators uploading to GCS needs to handle googleapiclient.errors
                 Key: AIRFLOW-4236
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4236
             Project: Apache Airflow
          Issue Type: New Feature
    Affects Versions: 1.10.3
            Reporter: jack


I got HTTP 500 error when using the MySqlToGoogleCloudStorageOperator this error caused the operator to fail and a 3 hours job were gone to waste.

This error should not cause the operator to fail immediately. It should first try to retry the upload as suggested : 

[https://stackoverflow.com/questions/23945784/how-to-manage-google-api-errors-in-python]

There should be at least 2 attempts to retry before giving up.

log:

INFO - Subtask: Traceback (most recent call last):
INFO - Subtask: File "/usr/local/bin/airflow", line 27, in <module>
INFO - Subtask: args.func(args)
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 392, in run
INFO - Subtask: pool=args.pool,
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 50, in wrapper
INFO - Subtask: result = func(*args, **kwargs)
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1493, in _run_raw_task
INFO - Subtask: result = task_copy.execute(context=context)
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/mysql_to_gcs.py", line 99, in execute
INFO - Subtask: self._upload_to_gcs(files_to_upload)
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/mysql_to_gcs.py", line 184, in _upload_to_gcs
INFO - Subtask: hook.upload(self.bucket, object, tmp_file_handle.name, 'application/json')
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcs_hook.py", line 131, in upload
INFO - Subtask: .insert(bucket=bucket, name=object, media_body=media) \
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/oauth2client/util.py", line 137, in positional_wrapper
INFO - Subtask: return wrapped(*args, **kwargs)
INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/googleapiclient/http.py", line 842, in execute
INFO - Subtask: raise HttpError(resp, content, uri=self.uri)
INFO - Subtask: googleapiclient.errors.HttpError: <HttpError 500 when requesting https://www.googleapis.com/upload/storage/v1/b/.......json returned "Backend Error">



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)