You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 18:16:52 UTC

[GitHub] [beam] damccorm opened a new issue, #20608: Support more granular splitting in BigQueryStorageStreamSource

damccorm opened a new issue, #20608:
URL: https://github.com/apache/beam/issues/20608

   The BigQuery storage API supports partitioning of streams in the API. However, the partitioning granularity is not sufficient for some pipelines, for which it is desirable to be able to dynamically rebalance down to the level of individual rows. This can be implemented by storing both the stream ID and an offset range as components of a stream source, as opposed to a stream ID only.
   
   Imported from Jira [BEAM-11215](https://issues.apache.org/jira/browse/BEAM-11215). Original Jira may contain additional context.
   Reported by: kjung520.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org