You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Duan Shiqiang (JIRA)" <ji...@apache.org> on 2018/12/27 13:55:00 UTC
[jira] [Updated] (AIRFLOW-3577) BigQueryToBigQueryOperator doesn't
work for BigQuery dataset in regions other than US and EU
[ https://issues.apache.org/jira/browse/AIRFLOW-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Duan Shiqiang updated AIRFLOW-3577:
-----------------------------------
Affects Version/s: 1.9.0
1.10.0
Component/s: operators
hooks
> BigQueryToBigQueryOperator doesn't work for BigQuery dataset in regions other than US and EU
> --------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3577
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3577
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, hooks, operators
> Affects Versions: 1.9.0, 1.10.0, 1.10.1
> Reporter: Duan Shiqiang
> Priority: Critical
>
> When using BigQueryToBigQueryOperator to copy a bigquery table to another table that resides in location other than US and EU, it will fail because can't query job status.
> When the operator runs the copy using Airflow bigquery hook here:
> [https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/bigquery_to_bigquery.py#L90]
> Eventually the bigquery hook pools the job by calling[ bigquery's get job API|https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/get]
> [https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L1092-L1094]
> The bigquery's get job API requires to pass the location parameter if the job is not in US or EU.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)