You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Valentyn Tymofieiev (JIRA)" <ji...@apache.org> on 2018/11/07 22:58:00 UTC
[jira] [Updated] (BEAM-6014) bigquery_tornadoes example does not
work on Python Direct Runner
[ https://issues.apache.org/jira/browse/BEAM-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Valentyn Tymofieiev updated BEAM-6014:
--------------------------------------
Description:
Steps to reproduce:
{noformat}
<Make new virtual env, install apache-beam[gcp] >
PROJECT=$(gcloud config get-value project)
BQ_DATASET=${USER}_bq_dataset
TABLE_NAME=out
bq rm -rf --project=$PROJECT $BQ_DATASET
bq mk --project=$PROJECT $BQ_DATASET
python -m apache_beam.examples.cookbook.bigquery_tornadoes --output $BQ_DATASET.$TABLE_NAME --project=$PROJECT
bq show $PROJECT:$BQ_DATASET.$TABLE_NAME
{noformat}
{noformat}
Last modified Schema Total Rows Total Bytes Expiration Time Partitioning Labels
----------------- --------------------------- ------------ ------------- ------------ ------------------- --------
07 Nov 14:45:23 |- month: integer 0 0
|- tornado_count: integer
{noformat}
The resulting Database is empty, however it has several rows if we run the example with Dataflow runner.
Older versions of SDK also seem to have this problem.
cc: [~chamikara], [~charleschen].
was:
Steps to reproduce:
{noformat}
<Make new virtual env, install apache-beam[gcp] >
PROJECT=$(gcloud config get-value project)
BQ_DATASET=${USER}_bq_dataset
TABLE_NAME=out
bq rm -rf --project=$PROJECT $BQ_DATASET
bq mk --project=$PROJECT $BQ_DATASET
python -m apache_beam.examples.cookbook.bigquery_tornadoes --output $BQ_DATASET.$TABLE_NAME --project=$PROJECT
bq show $PROJECT:$BQ_DATASET.$TABLE_NAME
{noformat}
The resulting Database is empty, however it has several rows if we run the example with Dataflow runner.
Older versions of SDK also seem to have this problem.
cc: [~chamikara], [~charleschen].
> bigquery_tornadoes example does not work on Python Direct Runner
> ----------------------------------------------------------------
>
> Key: BEAM-6014
> URL: https://issues.apache.org/jira/browse/BEAM-6014
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Ahmet Altay
> Priority: Major
>
> Steps to reproduce:
>
> {noformat}
> <Make new virtual env, install apache-beam[gcp] >
>
> PROJECT=$(gcloud config get-value project)
> BQ_DATASET=${USER}_bq_dataset
> TABLE_NAME=out
> bq rm -rf --project=$PROJECT $BQ_DATASET
> bq mk --project=$PROJECT $BQ_DATASET
> python -m apache_beam.examples.cookbook.bigquery_tornadoes --output $BQ_DATASET.$TABLE_NAME --project=$PROJECT
> bq show $PROJECT:$BQ_DATASET.$TABLE_NAME
> {noformat}
> {noformat}
> Last modified Schema Total Rows Total Bytes Expiration Time Partitioning Labels
> ----------------- --------------------------- ------------ ------------- ------------ ------------------- --------
> 07 Nov 14:45:23 |- month: integer 0 0
> |- tornado_count: integer
> {noformat}
>
> The resulting Database is empty, however it has several rows if we run the example with Dataflow runner.
> Older versions of SDK also seem to have this problem.
> cc: [~chamikara], [~charleschen].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)