You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/04/12 13:20:41 UTC
[jira] [Commented] (BEAM-1909) BigQuery read transform fails for
DirectRunner when querying non-US regions
[ https://issues.apache.org/jira/browse/BEAM-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965824#comment-15965824 ]
ASF GitHub Bot commented on BEAM-1909:
--------------------------------------
GitHub user ubunatic opened a pull request:
https://github.com/apache/beam/pull/2509
[BEAM-1909] BigQuery read transform fails for DirectRunner when querying non-US regions
I partially fixed the issue by getting the `location` of the source `Dataset`. Then I use this location as location of the created temp dataset. The added parameters are optional and should not break anything.
**Note:** The solution works for me when using the `DirectRunner` if I create a `BigQuerySource` with `table=<some-table>`. It does not work for a `BigQuerySource` with `query=<some-query>`.
The corresponding Jira issue should not yet be closed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ubunatic/beam temp-table-region
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2509.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2509
----
commit 79537846431fb679cde85755e4bbdd868cff3eee
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-10T12:24:44Z
better log message for bigquery temp tables
commit 652663887c3448046ede70d1f430940e1cfdd9ce
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-10T12:30:32Z
fixed spelling, improved wording
commit 4378aceeb9b3ee244452510fb5b729954cf52fea
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-10T15:14:48Z
use log formatting instead of string formatting
commit b818777335398fe93d26ffc118ddfc1609f7ddc0
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-11T15:10:26Z
Merge remote-tracking branch 'origin/master' into temp-table-log-message
commit 0075addfeab1777feb2e4aa7217936dd882bc52e
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-12T12:54:42Z
ignore cythonized files
commit 9f966e60bb29b65588a7aed5267d53e375853887
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-12T12:56:50Z
add temp dataset location for non-query BigQuerySource
commit 7d9f5ee0c1705073ae240805d972ad0c6ca07de4
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-12T13:01:50Z
Merge branch 'master' into temp-table-region
commit d773fbd44d9f6d8101aca7701702bae2563ee4a5
Author: Uwe Jugel <uw...@lovoo.com>
Date: 2017-04-12T13:03:42Z
fixed location comment
----
> BigQuery read transform fails for DirectRunner when querying non-US regions
> ---------------------------------------------------------------------------
>
> Key: BEAM-1909
> URL: https://issues.apache.org/jira/browse/BEAM-1909
> Project: Beam
> Issue Type: Bug
> Components: sdk-py
> Reporter: Chamikara Jayalath
>
> See: http://stackoverflow.com/questions/42135002/google-dataflow-cannot-read-and-write-in-different-locations-python-sdk-v0-5-5/42144748?noredirect=1#comment73621983_42144748
> This should be fixed by creating the temp dataset and table in the correct region.
> cc: [~sb2nov]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)