You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Robin, Martin (Nokia - IN/Bangalore)" <ma...@nokia.com> on 2016/12/19 13:02:10 UTC
Kafka controlled shutdown hangs when there are large number of topics
in the cluster
Hi
We have 9 broker instances in a kafka cluster spread across 3 linux machines. The 1st machine has 4 broker instances. 2nd machine has 4 broker instances and 3rd one has 1 broker instance. There are around 101 topics created in the cluster
We start the broker as follows
All 4 brokers are started on first machine
All 4 brokers are started on 2nd machine
1 broker started on 3rd machine
After brokers were running for sometime, we try to shutdown the brokers as below
All 4 brokers stopped on 1st machine
4 brokers are stopped on 2nd machine---- While we do this kafka controlled shutdown hangs
This same issue was not seen with 25 topics.
Please let us know if any solution is known to this issue
Thanks
Martin
Re: Kafka controlled shutdown hangs when there are large number of
topics in the cluster
Posted by James Cheng <wu...@gmail.com>.
> On Dec 19, 2016, at 10:13 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> Can you try setting auto.leader.rebalance.enable=false in your
> configuration (for all brokers) and see if it solves this problem?
> We've had some reports regarding this feature interfering with
> controlled shutdown.
>
Gwen, are there any JIRAs about this? Or can you point me to messages where I can read more about it?
If we did this, then we would have to manually run kafka-preferred-replica-election.sh after we're done restarting the cluster, right?
-James
> On Mon, Dec 19, 2016 at 5:02 AM, Robin, Martin (Nokia - IN/Bangalore)
> <ma...@nokia.com> wrote:
>> Hi
>>
>> We have 9 broker instances in a kafka cluster spread across 3 linux machines. The 1st machine has 4 broker instances. 2nd machine has 4 broker instances and 3rd one has 1 broker instance. There are around 101 topics created in the cluster
>>
>> We start the broker as follows
>> All 4 brokers are started on first machine
>> All 4 brokers are started on 2nd machine
>> 1 broker started on 3rd machine
>>
>> After brokers were running for sometime, we try to shutdown the brokers as below
>> All 4 brokers stopped on 1st machine
>> 4 brokers are stopped on 2nd machine---- While we do this kafka controlled shutdown hangs
>>
>> This same issue was not seen with 25 topics.
>>
>> Please let us know if any solution is known to this issue
>>
>> Thanks
>> Martin
>>
>>
>>
>>
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
Re: Kafka controlled shutdown hangs when there are large number of
topics in the cluster
Posted by Gwen Shapira <gw...@confluent.io>.
Can you try setting auto.leader.rebalance.enable=false in your
configuration (for all brokers) and see if it solves this problem?
We've had some reports regarding this feature interfering with
controlled shutdown.
On Mon, Dec 19, 2016 at 5:02 AM, Robin, Martin (Nokia - IN/Bangalore)
<ma...@nokia.com> wrote:
> Hi
>
> We have 9 broker instances in a kafka cluster spread across 3 linux machines. The 1st machine has 4 broker instances. 2nd machine has 4 broker instances and 3rd one has 1 broker instance. There are around 101 topics created in the cluster
>
> We start the broker as follows
> All 4 brokers are started on first machine
> All 4 brokers are started on 2nd machine
> 1 broker started on 3rd machine
>
> After brokers were running for sometime, we try to shutdown the brokers as below
> All 4 brokers stopped on 1st machine
> 4 brokers are stopped on 2nd machine---- While we do this kafka controlled shutdown hangs
>
> This same issue was not seen with 25 topics.
>
> Please let us know if any solution is known to this issue
>
> Thanks
> Martin
>
>
>
>
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog