You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/31 11:39:09 UTC

[GitHub] [airflow] totogo commented on issue #15088: GCP BigQuery Data Transfer Run Issue

totogo commented on issue #15088:
URL: https://github.com/apache/airflow/issues/15088#issuecomment-810996934


   > When this operator was created, Google only supported one location and then made a breaking change to the API to support multiple locations.
   > 
   > I think PR is necessary here to update to the new API.
   
   @mik-laj I see, right now I pass the location as part of `project_id` in the operator as a workaround, i.e. change `project_id=<project-id>` to `project_id=<project-id>/locations/<location>`, which works. 
   
   I can work on the PR. The proper solution should be to add a new argument `location_id` in the hooks and operators? or to add the full resouce name? Since it is already returned by the operator `BigQueryCreateDataTransferOperator` in Xcom `return_value`:
   ```
   {'name': 'projects/<project-id>/locations/europe/transferConfigs/<transfer-config-id>', 
   'destination_dataset_id': 'dest', 
   'display_name': 'Copy source to dest', 
   'update_time': '2021-03-31T07:31:54.655068Z', 
   'data_source_id': 'cross_region_copy',
   'params': {'overwrite_destination_table': True, 'source_dataset_id': 'source', 'source_project_id': '<source-project-id>'}, 'user_id': 'xxxx', 
   'dataset_region': 'europe', 
   'schedule_options': {'disable_auto_scheduling': True}, 
   'schedule': '', 'data_refresh_window_days': 0,
   'disabled': False, 
   'state': 0, 
   'notification_pubsub_topic': ''}
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org