You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by 堅強de泡沫 <je...@qq.com> on 2018/08/16 04:26:04 UTC
How to reduce kafka's rebalance time ?
hello:
How to reduce kafka's rebalance time ?
It takes a lot of time to rebalance each time. Why?
Re: How to reduce kafka's rebalance time ?
Posted by Steve Tian <st...@gmail.com>.
My $0.02:
1. Read the documentation.
2. Help people to understand your problem better: Try to describe your
problem in a gist and see if you can provide some details like: which
version of Kafka you're using on client/server side, your
consumer/producer/server code/configuration that can reproduce your
problem.
3. Help yourself to understand your problem better: Is the problem still
reproducible in different version/cluster/configuration/process logic? For
example, you found there is a 10 minutes delay, what kind of configuration
is 10 minutes related? Will delay get changed when you change your
configurations? How long does it take to process records returned from a
single poll?
4. Learn from your logging/metrics: If you can reproduce the problem, try
to see if anything unusual in the logs/metrics. Try too enable/add logging
in your consumer, consumer rebalance listener and interceptor. Try to
close your consumer gracefully/abruptly, try to kill your Java consumer
process, or even try to pause your Java consumer process to simulate gc
pause. You should find something in the logs and JMX metrics, try to
reason about them. You will need this for production
troubleshooting/monitoring.
Cheers,
Steve
On Thu, Aug 16, 2018, 7:48 PM Shantanu Deshmukh <sh...@gmail.com>
wrote:
> Hi Manna,
>
> I meant no offense. Simply meant to say that haven't found solution to my
> problem from here.
> Apologies, if my sentence was off the line.
>
> On Thu, Aug 16, 2018 at 4:05 PM M. Manna <ma...@gmail.com> wrote:
>
> > You have been recommended to upgrade to a newer version of Kafka, or tune
> > timeout params. Adhering to a older version is more of the users’
> decision.
> > Perhaps, we should simply put older versions as “End of Life”.
> >
> > As part of open source initiative, you are always welcome to debug and
> > demonstrate how your use case is different, and raise a KIP.
> >
> > Not sure what you mean by “*no have I received any help from here.” *
> >
> > We are always actively trying to contribute as much as we can, and
> > sometimes the answers may not be according to your expectations or
> > timeline. Hence, the open source initiative.
> >
> > Hope this makes sense.
> >
> > Regards,
> >
> > Regards,
> > On Thu, 16 Aug 2018 at 06:55, Shantanu Deshmukh <sh...@gmail.com>
> > wrote:
> >
> > > I am also facing the same issue. Whenever I am restarting my consumers
> it
> > > is taking upto 10 minutes to start consumption. Also some of the
> > consumers
> > > randomly rebalance and it again takes same amount of time to complete
> > > rebalance.
> > > I haven't been able to figure out any solution for this issue, nor
> have I
> > > received any help from here.
> > >
> > > On Thu, Aug 16, 2018 at 9:56 AM 堅強de泡沫 <je...@qq.com> wrote:
> > >
> > > > hello:
> > > > How to reduce kafka's rebalance time ?
> > > > It takes a lot of time to rebalance each time. Why?
> > >
> >
>
Re: How to reduce kafka's rebalance time ?
Posted by Shantanu Deshmukh <sh...@gmail.com>.
Hi Manna,
I meant no offense. Simply meant to say that haven't found solution to my
problem from here.
Apologies, if my sentence was off the line.
On Thu, Aug 16, 2018 at 4:05 PM M. Manna <ma...@gmail.com> wrote:
> You have been recommended to upgrade to a newer version of Kafka, or tune
> timeout params. Adhering to a older version is more of the users’ decision.
> Perhaps, we should simply put older versions as “End of Life”.
>
> As part of open source initiative, you are always welcome to debug and
> demonstrate how your use case is different, and raise a KIP.
>
> Not sure what you mean by “*no have I received any help from here.” *
>
> We are always actively trying to contribute as much as we can, and
> sometimes the answers may not be according to your expectations or
> timeline. Hence, the open source initiative.
>
> Hope this makes sense.
>
> Regards,
>
> Regards,
> On Thu, 16 Aug 2018 at 06:55, Shantanu Deshmukh <sh...@gmail.com>
> wrote:
>
> > I am also facing the same issue. Whenever I am restarting my consumers it
> > is taking upto 10 minutes to start consumption. Also some of the
> consumers
> > randomly rebalance and it again takes same amount of time to complete
> > rebalance.
> > I haven't been able to figure out any solution for this issue, nor have I
> > received any help from here.
> >
> > On Thu, Aug 16, 2018 at 9:56 AM 堅強de泡沫 <je...@qq.com> wrote:
> >
> > > hello:
> > > How to reduce kafka's rebalance time ?
> > > It takes a lot of time to rebalance each time. Why?
> >
>
Re: How to reduce kafka's rebalance time ?
Posted by "M. Manna" <ma...@gmail.com>.
You have been recommended to upgrade to a newer version of Kafka, or tune
timeout params. Adhering to a older version is more of the users’ decision.
Perhaps, we should simply put older versions as “End of Life”.
As part of open source initiative, you are always welcome to debug and
demonstrate how your use case is different, and raise a KIP.
Not sure what you mean by “*no have I received any help from here.” *
We are always actively trying to contribute as much as we can, and
sometimes the answers may not be according to your expectations or
timeline. Hence, the open source initiative.
Hope this makes sense.
Regards,
Regards,
On Thu, 16 Aug 2018 at 06:55, Shantanu Deshmukh <sh...@gmail.com>
wrote:
> I am also facing the same issue. Whenever I am restarting my consumers it
> is taking upto 10 minutes to start consumption. Also some of the consumers
> randomly rebalance and it again takes same amount of time to complete
> rebalance.
> I haven't been able to figure out any solution for this issue, nor have I
> received any help from here.
>
> On Thu, Aug 16, 2018 at 9:56 AM 堅強de泡沫 <je...@qq.com> wrote:
>
> > hello:
> > How to reduce kafka's rebalance time ?
> > It takes a lot of time to rebalance each time. Why?
>
Re: How to reduce kafka's rebalance time ?
Posted by Steve Tian <st...@gmail.com>.
Interesting. Thanks for letting us know.
On Fri, 17 Aug 2018 at 11:22 堅強de泡沫 <je...@qq.com> wrote:
> I modified some consumer configuration items, mainly about the parameters
> of fetch frequency, heartbeat, and session timeout. The problem of long
> time rebalance has not been found in the test environment for a long time.
> The relevant configuration is as follows:fetch-max-wait: 1s
> heartbeat-interval: 1s
> session.timeout.ms: 10000
> metadata.max.age.ms: 6000
> max.poll.records: 100
> max.poll.interval.ms: 500
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Shantanu Deshmukh"<sh...@gmail.com>;
> 发送时间: 2018年8月16日(星期四) 下午2:25
> 收件人: "users"<us...@kafka.apache.org>;
>
> 主题: Re: How to reduce kafka's rebalance time ?
>
>
>
> I am also facing the same issue. Whenever I am restarting my consumers it
> is taking upto 10 minutes to start consumption. Also some of the consumers
> randomly rebalance and it again takes same amount of time to complete
> rebalance.
> I haven't been able to figure out any solution for this issue, nor have I
> received any help from here.
>
> On Thu, Aug 16, 2018 at 9:56 AM 堅強de泡沫 <je...@qq.com> wrote:
>
> > hello:
> > How to reduce kafka's rebalance time ?
> > It takes a lot of time to rebalance each time. Why?
回复： How to reduce kafka's rebalance time ?
Posted by 堅強de泡沫 <je...@qq.com>.
I modified some consumer configuration items, mainly about the parameters of fetch frequency, heartbeat, and session timeout. The problem of long time rebalance has not been found in the test environment for a long time. The relevant configuration is as follows:fetch-max-wait: 1s
heartbeat-interval: 1s
session.timeout.ms: 10000
metadata.max.age.ms: 6000
max.poll.records: 100
max.poll.interval.ms: 500
------------------ 原始邮件 ------------------
发件人: "Shantanu Deshmukh"<sh...@gmail.com>;
发送时间: 2018年8月16日(星期四) 下午2:25
收件人: "users"<us...@kafka.apache.org>;
主题: Re: How to reduce kafka's rebalance time ?
I am also facing the same issue. Whenever I am restarting my consumers it
is taking upto 10 minutes to start consumption. Also some of the consumers
randomly rebalance and it again takes same amount of time to complete
rebalance.
I haven't been able to figure out any solution for this issue, nor have I
received any help from here.
On Thu, Aug 16, 2018 at 9:56 AM 堅強de泡沫 <je...@qq.com> wrote:
> hello:
> How to reduce kafka's rebalance time ?
> It takes a lot of time to rebalance each time. Why?
Re: How to reduce kafka's rebalance time ?
Posted by Shantanu Deshmukh <sh...@gmail.com>.
I am also facing the same issue. Whenever I am restarting my consumers it
is taking upto 10 minutes to start consumption. Also some of the consumers
randomly rebalance and it again takes same amount of time to complete
rebalance.
I haven't been able to figure out any solution for this issue, nor have I
received any help from here.
On Thu, Aug 16, 2018 at 9:56 AM 堅強de泡沫 <je...@qq.com> wrote:
> hello:
> How to reduce kafka's rebalance time ?
> It takes a lot of time to rebalance each time. Why?