[jira] [Created] (BEAM-10926) Specify the event time when consuming pubsub data.

["Jian Zheng (Jira)" <ji...@apache.org>]

Jian Zheng created BEAM-10926:
---------------------------------

             Summary: Specify the event time when consuming pubsub data.
                 Key: BEAM-10926
                 URL: https://issues.apache.org/jira/browse/BEAM-10926
             Project: Beam
          Issue Type: Improvement
          Components: io-java-gcp
            Reporter: Jian Zheng


I need to specify the event time when consuming pubsub data.
{code:java}
PCollection<PubsubMessage> pubsubMessages = pipeline.apply("Read Pub/Sub Data",
       PubsubIO.readMessagesWithAttributes()
                .withTimestampAttribute(options.getTimeAttribute())
                .fromSubscription(options.getInputSubscription()));
{code}
The only way to do this is to use the {color:#FF0000}withTimestampAttribute(){color} method.

But if  I use a timestamp in some other format, such as a 19-bit nanosecond, or if save the event time in the payload. The method won't work !

So I had to extend the PubsubClient class and override the PubsubClient. extractTimestamp() method.

I'am hoping to provide a way to pass in some implementation class that would allow to parse out timestamp from the current pubsub message.

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)