You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/02 10:41:37 UTC

[GitHub] [airflow] potiuk commented on pull request #9590: Improve idempotency of BigQueryInsertJobOperator

potiuk commented on pull request #9590:
URL: https://github.com/apache/airflow/pull/9590#issuecomment-667657549


   Very interesting discussion :).
   
   I really like the proposal of @jaketf - with the explicitness of it (all these options are merely optimisations at the expense of predictability), but I wanted to modify it slightly. I believe there is no need to limit ourselves to the states defined in https://cloud.google.com/bigquery/docs/reference/auditlogs/rest/Shared.Types/JobState. With 4 states, we can easily define our own states:
   
   - PENDING
   - RUNNING
   - SUCCESS
   - FAILURE
   
   This will be far less confusing than additional `retry_on_reattach_to_error` parameter. 
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org