You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Daniel Halperin (JIRA)" <ji...@apache.org> on 2016/02/25 15:22:18 UTC

[jira] [Created] (BEAM-46) Unbounded sink for Google Cloud Bigtable

Daniel Halperin created BEAM-46:
-----------------------------------

             Summary: Unbounded sink for Google Cloud Bigtable
                 Key: BEAM-46
                 URL: https://issues.apache.org/jira/browse/BEAM-46
             Project: Beam
          Issue Type: New Feature
          Components: sdk-java-gcp
            Reporter: Daniel Halperin
            Assignee: Daniel Halperin


Google Cloud Bigtable is currently in Beta. https://cloud.google.com/bigtable/ A bounded sink is included in the initial code for Beam, and uses asynchronous row mutations (with bounded memory) for maximum throughput.

The unbounded sink code is in principle not too different. The key areas of focus are better connection management, thread management, and fault tolerance (e.g., so connections are not leaked if bundles fail) in the unbounded case in which there are hundreds of active threads and very small bundles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)