You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/05/06 07:23:00 UTC
[jira] [Work logged] (BEAM-4824) Get BigQueryIO batch loads to
return something actionable
[ https://issues.apache.org/jira/browse/BEAM-4824?focusedWorklogId=431064&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-431064 ]
ASF GitHub Bot logged work on BEAM-4824:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/May/20 07:22
Start Date: 06/May/20 07:22
Worklog Time Spent: 10m
Work Description: giogit commented on pull request #6055:
URL: https://github.com/apache/beam/pull/6055#issuecomment-624484946
This pull request is very important, in my opinion.
With other connectors, like JdbcIO or TextIO, it is possible to use Wait.on correctly and I think this is a useful feature for batch pipelines.
At the moment, using BigQueryIO, the Wait.on works correctly only with streaming inserts.
I use Wait.on to send PubSub messages at the end of my batch pipelines with JdbcIO and I'd like to do the same also with BigQueryIO
Thank you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 431064)
Time Spent: 2.5h (was: 2h 20m)
> Get BigQueryIO batch loads to return something actionable
> ---------------------------------------------------------
>
> Key: BEAM-4824
> URL: https://issues.apache.org/jira/browse/BEAM-4824
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Carlos Alonso
> Assignee: Carlos Alonso
> Priority: Minor
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> ATM BigQueryIO batchloads returns an empty collection that has no information related to how the load job finished. It is even returned before the job finishes.
>
> Change it so that:
> # The returning PCollection only appers when the job has actually finished
> # The returning PCollection contains information about the job result
--
This message was sent by Atlassian Jira
(v8.3.4#803005)