[jira] [Created] (BEAM-53) PubSubIO: reimplement in Java

["Daniel Halperin (JIRA)" <ji...@apache.org>]

Daniel Halperin created BEAM-53:
-----------------------------------

             Summary: PubSubIO: reimplement in Java
                 Key: BEAM-53
                 URL: https://issues.apache.org/jira/browse/BEAM-53
             Project: Beam
          Issue Type: New Feature
          Components: sdk-java-gcp
            Reporter: Daniel Halperin
            Priority: Minor


PubSubIO is currently only partially implemented in Java: the DirectPipelineRunner uses a non-scalable API in a single-threaded manner.

In contrast, the DataflowPipelineRunner uses an entirely different code path implemented in the Google Cloud Dataflow service.

We need to reimplement PubSubIO in Java in order to support other runners in a scalable way.

Additionally, we can take this opportunity to add new features:

* getting timestamp from an arbitrary lambda in arbitrary formats rather than from a message attribute in only 2 formats.
* exposing metadata and attributes in the elements produced by PubSubIO.Read
* setting metadata and attributes in the messages written by PubSubIO.Write




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)