You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "1095193290@qq.com" <10...@qq.com> on 2018/01/09 06:44:58 UTC
Need feedbacks to consumer if hangs due to some __consumer_offsets partitions failed
hi
My kafka cluster has two brokers( 0 and 1) and it created __consumer_offsets topic has 50 partitions . When broker 0 failed, half of partitions failed (because topic __consumer_offsets ReplicationFactor is 1).
This results consequence that my kafkaConsumer sometimes can work well and sometimes hangs with 0 outputs. My consumer generated a new goupid upon it start up . When consumer startup, it will request groupcoordinator with groupid . When Consumer's groupid is routed to a good partition, it can work well, when grouid is routed to a failed partition, it hangs with a response GROUP_COORDINATOR_NOT_AVAILABLE. I hope broker can feedback to consumer if hangs due to some __consumer_offsets partitions failed , it can help user improve expericence and debug issure fastly.
1095193290@qq.com
Re: Need feedbacks to consumer if hangs due to some
__consumer_offsets partitions failed
Posted by Xin Li <Xi...@trivago.com>.
Hey,
I think you can do it in two ways:
1. Manually increase the replication factor
2. In the server.config file, change the default.replication.factor
Best,
Xin
On 09.01.18, 07:46, "1095193290@qq.com" <10...@qq.com> wrote:
hi
My kafka cluster has two brokers( 0 and 1) and it created __consumer_offsets topic has 50 partitions . When broker 0 failed, half of partitions failed (because topic __consumer_offsets ReplicationFactor is 1).
This results consequence that my kafkaConsumer sometimes can work well and sometimes hangs with 0 outputs. My consumer generated a new goupid upon it start up . When consumer startup, it will request groupcoordinator with groupid . When Consumer's groupid is routed to a good partition, it can work well, when grouid is routed to a failed partition, it hangs with a response GROUP_COORDINATOR_NOT_AVAILABLE. I hope broker can feedback to consumer if hangs due to some __consumer_offsets partitions failed , it can help user improve expericence and debug issure fastly.
1095193290@qq.com