You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Ryan Williams <rw...@gmail.com> on 2014/08/14 20:10:14 UTC
How to perform a controlled shutdown for rolling bounce?
Running 0.8.1 and am unable to do a controlled shutdown as part of a
rolling bounce.
Is this the primary reference for this task?
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
I've set the config to enable controlled shutdown.
controlled.shutdown.enable=true
controlled.shutdown.max.retries=3
controlled.shutdown.retry.backoff.ms=5000
Before shutting down the first broker, topics looks like:
Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
Topic: events Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
Topic: failure Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
I then executed the bin/kafka-server-stop.sh program.
After that, the topics look like:
Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
Topic: events Partition: 0 Leader: -1 Replicas: 1 Isr:
Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
Topic: failure Partition: 0 Leader: -1 Replicas: 1 Isr:
Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
What does the -1 for Leader and blank Isr indicate? Do I need to run
something else for the leader election to occur? I thought that was
automatic with the controlled shutdown enabled. Is there a different
shutdown command to issue?
Thanks!
Ryan
Re: How to perform a controlled shutdown for rolling bounce?
Posted by Joel Koshy <jj...@gmail.com>.
Yes - you would still issue SIGTERM. Also, right now you will need to
use the following process to increase replication factor:
http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
There is a jira open to help reduce the effort to do that.
Joel
On Thu, Aug 14, 2014 at 12:59:16PM -0700, Ryan Williams wrote:
> Thanks for clarifying.
>
> When I increase the replication factor, enable controlled shutdown and want
> to do a controlled shutdown, do I still issue the same shutdown (SIGTERM)?
>
>
> On Thu, Aug 14, 2014 at 11:40 AM, Joel Koshy <jj...@gmail.com> wrote:
>
> > Controlled shutdown does not really help in your case since your
> > replication factor is one.
> >
> > > What does the -1 for Leader and blank Isr indicate? Do I need to run
> >
> > It means the partition is unavailable (since there are no other
> > replicas).
> >
> > So you should either use a higher replication factor or just turn off
> > controlled shutdown and do a regular shutdown (SIGTERM) in which case
> > the partition will remain unavailable while the broker is down.
> >
> >
> > Joel
> >
> > On Thu, Aug 14, 2014 at 11:10:14AM -0700, Ryan Williams wrote:
> > > Running 0.8.1 and am unable to do a controlled shutdown as part of a
> > > rolling bounce.
> > >
> > > Is this the primary reference for this task?
> > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
> > >
> > > I've set the config to enable controlled shutdown.
> > >
> > > controlled.shutdown.enable=true
> > > controlled.shutdown.max.retries=3
> > > controlled.shutdown.retry.backoff.ms=5000
> > >
> > > Before shutting down the first broker, topics looks like:
> > >
> > > Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
> > > Topic: events Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> > > Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > > Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
> > > Topic: failure Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> > > Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > > Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
> > > Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
> > > Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
> > >
> > >
> > > I then executed the bin/kafka-server-stop.sh program.
> > >
> > > After that, the topics look like:
> > >
> > > Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
> > > Topic: events Partition: 0 Leader: -1 Replicas: 1 Isr:
> > > Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > > Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
> > > Topic: failure Partition: 0 Leader: -1 Replicas: 1 Isr:
> > > Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > > Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
> > > Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
> > > Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
> > >
> > >
> > > What does the -1 for Leader and blank Isr indicate? Do I need to run
> > > something else for the leader election to occur? I thought that was
> > > automatic with the controlled shutdown enabled. Is there a different
> > > shutdown command to issue?
> > >
> > > Thanks!
> > > Ryan
> >
> >
Re: How to perform a controlled shutdown for rolling bounce?
Posted by Ryan Williams <rw...@gmail.com>.
Thanks for clarifying.
When I increase the replication factor, enable controlled shutdown and want
to do a controlled shutdown, do I still issue the same shutdown (SIGTERM)?
On Thu, Aug 14, 2014 at 11:40 AM, Joel Koshy <jj...@gmail.com> wrote:
> Controlled shutdown does not really help in your case since your
> replication factor is one.
>
> > What does the -1 for Leader and blank Isr indicate? Do I need to run
>
> It means the partition is unavailable (since there are no other
> replicas).
>
> So you should either use a higher replication factor or just turn off
> controlled shutdown and do a regular shutdown (SIGTERM) in which case
> the partition will remain unavailable while the broker is down.
>
>
> Joel
>
> On Thu, Aug 14, 2014 at 11:10:14AM -0700, Ryan Williams wrote:
> > Running 0.8.1 and am unable to do a controlled shutdown as part of a
> > rolling bounce.
> >
> > Is this the primary reference for this task?
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
> >
> > I've set the config to enable controlled shutdown.
> >
> > controlled.shutdown.enable=true
> > controlled.shutdown.max.retries=3
> > controlled.shutdown.retry.backoff.ms=5000
> >
> > Before shutting down the first broker, topics looks like:
> >
> > Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: events Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> > Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: failure Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> > Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
> > Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
> >
> >
> > I then executed the bin/kafka-server-stop.sh program.
> >
> > After that, the topics look like:
> >
> > Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: events Partition: 0 Leader: -1 Replicas: 1 Isr:
> > Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: failure Partition: 0 Leader: -1 Replicas: 1 Isr:
> > Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> > Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
> > Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
> >
> >
> > What does the -1 for Leader and blank Isr indicate? Do I need to run
> > something else for the leader election to occur? I thought that was
> > automatic with the controlled shutdown enabled. Is there a different
> > shutdown command to issue?
> >
> > Thanks!
> > Ryan
>
>
Re: How to perform a controlled shutdown for rolling bounce?
Posted by Joel Koshy <jj...@gmail.com>.
Controlled shutdown does not really help in your case since your
replication factor is one.
> What does the -1 for Leader and blank Isr indicate? Do I need to run
It means the partition is unavailable (since there are no other
replicas).
So you should either use a higher replication factor or just turn off
controlled shutdown and do a regular shutdown (SIGTERM) in which case
the partition will remain unavailable while the broker is down.
Joel
On Thu, Aug 14, 2014 at 11:10:14AM -0700, Ryan Williams wrote:
> Running 0.8.1 and am unable to do a controlled shutdown as part of a
> rolling bounce.
>
> Is this the primary reference for this task?
>
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
>
> I've set the config to enable controlled shutdown.
>
> controlled.shutdown.enable=true
> controlled.shutdown.max.retries=3
> controlled.shutdown.retry.backoff.ms=5000
>
> Before shutting down the first broker, topics looks like:
>
> Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: events Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: failure Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
> Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
>
>
> I then executed the bin/kafka-server-stop.sh program.
>
> After that, the topics look like:
>
> Topic:events PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: events Partition: 0 Leader: -1 Replicas: 1 Isr:
> Topic: events Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> Topic:failure PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: failure Partition: 0 Leader: -1 Replicas: 1 Isr:
> Topic: failure Partition: 1 Leader: 2 Replicas: 2 Isr: 2
> Topic:retry PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: retry Partition: 0 Leader: 2 Replicas: 2 Isr: 2
> Topic: retry Partition: 1 Leader: 3 Replicas: 3 Isr: 3
>
>
> What does the -1 for Leader and blank Isr indicate? Do I need to run
> something else for the leader election to occur? I thought that was
> automatic with the controlled shutdown enabled. Is there a different
> shutdown command to issue?
>
> Thanks!
> Ryan