You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Mcs Vemuri <mc...@yahoo.com.INVALID> on 2023/03/09 00:52:27 UTC
max.poll.interval.ms rebalance override
Hello,
Is there a way to override a rebalance caused by max.poll.interval.ms? We have a case where there are multiple consumers in the same group- but some of them might start sooner than the rest and all consumers can be lazy pollers
The issue is that we need to set the poll interval to a large value as the consumer may not always need to poll- instead, they only poll when they receive some other stimulus. Because of this, when the other consumer joins the group, rebalance takes very long
Ideally, the rebalance timeout would have its own config(maybe max.rebalance.ms?) which defaults to max.poll.interval.ms but not sure if that was already considered and rejected due to some other reason
Re: max.poll.interval.ms rebalance override
Posted by Mcs Vemuri <mc...@yahoo.com.INVALID>.
I overlooked the pause resume mechanism- it should work for us I think
On Wednesday, March 8, 2023, 4:52 PM, Mcs Vemuri <mc...@yahoo.com> wrote:
Hello,
Is there a way to override a rebalance caused by max.poll.interval.ms? We have a case where there are multiple consumers in the same group- but some of them might start sooner than the rest and all consumers can be lazy pollers
The issue is that we need to set the poll interval to a large value as the consumer may not always need to poll- instead, they only poll when they receive some other stimulus. Because of this, when the other consumer joins the group, rebalance takes very long
Ideally, the rebalance timeout would have its own config(maybe max.rebalance.ms?) which defaults to max.poll.interval.ms but not sure if that was already considered and rejected due to some other reason
Re: max.poll.interval.ms rebalance override
Posted by Mcs Vemuri <mc...@yahoo.com.INVALID>.
I overlooked the pause resume mechanism- it should work for us I think
On Wednesday, March 8, 2023, 4:52 PM, Mcs Vemuri <mc...@yahoo.com> wrote:
Hello,
Is there a way to override a rebalance caused by max.poll.interval.ms? We have a case where there are multiple consumers in the same group- but some of them might start sooner than the rest and all consumers can be lazy pollers
The issue is that we need to set the poll interval to a large value as the consumer may not always need to poll- instead, they only poll when they receive some other stimulus. Because of this, when the other consumer joins the group, rebalance takes very long
Ideally, the rebalance timeout would have its own config(maybe max.rebalance.ms?) which defaults to max.poll.interval.ms but not sure if that was already considered and rejected due to some other reason