You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Daniel Mills (JIRA)" <ji...@apache.org> on 2016/06/28 20:07:57 UTC

[jira] [Created] (BEAM-384) Streaming BigQueryIO should support user-provided IDs

Daniel Mills created BEAM-384:
---------------------------------

             Summary: Streaming BigQueryIO should support user-provided IDs
                 Key: BEAM-384
                 URL: https://issues.apache.org/jira/browse/BEAM-384
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-gcp
    Affects Versions: 0.1.0-incubating, 0.2.0-incubating
            Reporter: Daniel Mills
            Assignee: Daniel Halperin
            Priority: Minor


Currently, BigQueryIO always assigns IDs and does a shuffle to ensure that they are atomic.  This incurs a noticeable cost and is unnecessary if the user already has deterministic IDs that they can use.  The sink should be able to use these IDs to skip the shuffle.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)