You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Thinh Ha (Jira)" <ji...@apache.org> on 2020/05/30 18:46:00 UTC
[jira] [Comment Edited] (BEAM-3489) Expose the message id of
received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120343#comment-17120343 ]
Thinh Ha edited comment on BEAM-3489 at 5/30/20, 6:45 PM:
----------------------------------------------------------
Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change?
In this PR the parseFn is always non-null, e.g. [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-a72606daf95658d0bb7c365b0d08416cR120]
which should trigger this branch of code that returns the PubsubMessage
[https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/PubsubReader.java#L120]
I can think of changing setNeedsAttributes to true in
[https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR472]
However, it appears the test against this did not fail?
[https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-598fd709155261a38592b0612939551aR73]
was (Author: thinhha):
Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change?
Looking at
> Expose the message id of received messages within PubsubMessage
> ---------------------------------------------------------------
>
> Key: BEAM-3489
> URL: https://issues.apache.org/jira/browse/BEAM-3489
> Project: Beam
> Issue Type: New Feature
> Components: io-java-gcp
> Reporter: Luke Cwik
> Assignee: Thinh Ha
> Priority: P3
> Labels: newbie, starter
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> This task is about passing forward the message id from the pubsub proto to the java PubsubMessage.
> Add a message id field to PubsubMessage.
> Update the coder for PubsubMessage to encode the message id.
> Update the translation from the Pubsub proto message to the Dataflow message:
> https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976
--
This message was sent by Atlassian Jira
(v8.3.4#803005)