[GitHub] yohei1126 commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook

[GitBox <gi...@apache.org>]

yohei1126 commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook
URL: https://github.com/apache/incubator-airflow/pull/4324#discussion_r244113891
 
 

 ##########
 File path: airflow/contrib/hooks/bigquery_hook.py
 ##########
 @@ -580,11 +587,18 @@ def run_query(self,
             by one or more columns. This is only available in combination with
             time_partitioning. The order of columns given determines the sort order.
         :type cluster_fields: list of str
+        :param location: The geographic location of the job. Required except for
+            US and EU. See details at
 
 Review comment:
   Hi, I tried to import to a data set in `asia-northeast1` using `GoogleCloudStorageToBigQueryOperator ` and I got the following error as kaxil mentioned.
   ```[2018-12-26 21:35:47,464] {base_task_runner.py:107} INFO - Job 146: Subtask bq_load_data_into_dest_table_from_gcs [2018-12-26 21:35:47,464] {discovery.py:871} INFO - URL being requested: GET https://www.googleapis.com/bigquery/v2/projects/my-project/jobs/job_abc123?alt=json
   [2018-12-26 21:35:47,931] {models.py:1736} ERROR - ('BigQuery job status check failed. Final error was: %s', 404)
   Traceback (most recent call last)
     File "/usr/local/lib/airflow/airflow/contrib/hooks/bigquery_hook.py", line 981, in run_with_configuratio
       jobId=self.running_job_id).execute(
     File "/usr/local/lib/python3.6/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrappe
       return wrapped(*args, **kwargs
     File "/usr/local/lib/python3.6/site-packages/googleapiclient/http.py", line 851, in execut
       raise HttpError(resp, content, uri=self.uri
   googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/bigquery/v2/projects/my-project/jobs/job_abc123?alt=json returned "Not found: Job my-project:job_abc123"
   
   During handling of the above exception, another exception occurred
   
   Traceback (most recent call last)
     File "/usr/local/lib/airflow/airflow/models.py", line 1633, in _run_raw_tas
       result = task_copy.execute(context=context
     File "/usr/local/lib/airflow/airflow/contrib/operators/gcs_to_bq.py", line 237, in execut
       time_partitioning=self.time_partitioning
     File "/usr/local/lib/airflow/airflow/contrib/hooks/bigquery_hook.py", line 951, in run_loa
       return self.run_with_configuration(configuration
     File "/usr/local/lib/airflow/airflow/contrib/hooks/bigquery_hook.py", line 1003, in run_with_configuratio
       err.resp.status
   Exception: ('BigQuery job status check failed. Final error was: %s', 404
   ```
   https://issues.apache.org/jira/browse/AIRFLOW-3571
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services