[GitHub] [beam] kennknowles opened a new issue, #18638: MQTT IO should compute watermark and ack messages outside of finalizeCheckpoint method

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #18638:
URL: https://github.com/apache/beam/issues/18638

   I'm experiencing a situation when an incoming message isn't acknowledged (therefore in sometime broker resend it) and the watermark is not updated while new messages are coming continuously.
   
   After some time I've discovered that this situation is related to the fact that finalizaCheckpoint is not being called.
   
   I took a look at the Pubsub IO implementation and found that they expect such situation and do not compute watermark and ack messages in finalizeCheckpoint. Here is the comment about that: [https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L289](https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L289)
   
   Should MQTT IO do the same?
   
   Imported from Jira [BEAM-3696](https://issues.apache.org/jira/browse/BEAM-3696). Original Jira may contain additional context.
   Reported by: kolchinmax.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org