You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 22:30:31 UTC
[GitHub] [beam] damccorm opened a new issue, #21259: Consumer group with random prefix
damccorm opened a new issue, #21259:
URL: https://github.com/apache/beam/issues/21259
I'm running two beam pipelines on Apache Flink which read messages from the same Kafka topic by using KafkaIO. Both of them are in the same consumer group, so in my understanding each message should be consumed by either pipeline (but not both) but I observed that the same set of messages is consumed by each pipeline, meaning each message is processed at least twice.
I can see in the log that additional consumer group with prefix "Reader-1_offset_consumer_<random_number\>_" was created and also consume messages.
Could you explain what should I do to consume messages only once by one consumer group and why that additional group was created?
Similar issue: BEAM-12880
Imported from Jira [BEAM-12995](https://issues.apache.org/jira/browse/BEAM-12995). Original Jira may contain additional context.
Reported by: htsi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] manuzhang commented on issue #21259: Consumer group with random prefix
Posted by GitBox <gi...@apache.org>.
manuzhang commented on issue #21259:
URL: https://github.com/apache/beam/issues/21259#issuecomment-1174048479
Duplicate of https://github.com/apache/beam/issues/20159 per original discussion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] manuzhang closed issue #21259: Consumer group with random prefix
Posted by GitBox <gi...@apache.org>.
manuzhang closed issue #21259: Consumer group with random prefix
URL: https://github.com/apache/beam/issues/21259
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org