You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Mingjie Lai <ml...@apache.org> on 2015/03/25 19:15:17 UTC

Plan of Controlled Shutdown

Hi.

I've been trying to figure out the best way to do kafka broker rolling
restart, and read the controlled shutdown wiki page:

https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown

However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:

/usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker

Exception in thread "main" java.lang.NoClassDefFoundError:
kafka/admin/ShutdownBroker
Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
        at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: kafka.admin.ShutdownBroker. Program will
exit.

Some other commands are there:

/usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
grep test

test

However, I also see some jira about the bug fixes for ControlledShutdown
like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
confused.

What's the plan for this feature in kafka? I still see the value of it, for
example, we can move some the leader partitions around on purpose for a
scheduled upgrade or config update, minimize the transition as less as
possible.

Thanks,
Mingjie

Re: Plan of Controlled Shutdown

Posted by Neha Narkhede <ne...@confluent.io>.
Removed the mention of the controlled shutdown tool from the wiki since we
don't support it anymore -
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown

On Wed, Mar 25, 2015 at 3:07 PM, Guozhang Wang <wa...@gmail.com> wrote:

> Mingjie,
>
> Controlled shutdown has been fully implemented and is regularly exercised
> at LinkedIn for version upgrades / rolling bounce tests, etc. As Harsha
> said you just need to turn on the config in the brokers, and normal
> shutting down command like control-c will trigger the controlled shutdown.
>
> Guozhang
>
> On Wed, Mar 25, 2015 at 12:39 PM, Harsha <ka...@harsha.io> wrote:
>
> > Hi ,
> >        you can enable/disable controlled shutdown using
> > controlled.shutdown.enable in server.properties. This is by default set
> to
> > “true” in 0.8.2. You can go ahead do a rolling restart  and you don’t
> need
> > ShutdownBroker command which is removed.  Regarding KAFKA-2029 talks
> about
> > improvement to the controlled shutdown especially in case of having high
> > partitions per broker which might degrade the performance as per the
> JIRA.
> >
> > Thanks,
> > Harsha
> >
> >
> > On March 25, 2015 at 11:15:21 AM, Mingjie Lai (mlai@apache.org) wrote:
> >
> > Hi.
> >
> > I've been trying to figure out the best way to do kafka broker rolling
> > restart, and read the controlled shutdown wiki page:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
> >
> > However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:
> >
> > /usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > kafka/admin/ShutdownBroker
> > Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> > Could not find the main class: kafka.admin.ShutdownBroker. Program will
> > exit.
> >
> > Some other commands are there:
> >
> > /usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
> > grep test
> >
> > test
> >
> > However, I also see some jira about the bug fixes for ControlledShutdown
> > like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
> > confused.
> >
> > What's the plan for this feature in kafka? I still see the value of it,
> for
> > example, we can move some the leader partitions around on purpose for a
> > scheduled upgrade or config update, minimize the transition as less as
> > possible.
> >
> > Thanks,
> > Mingjie
> >
>
>
>
> --
> -- Guozhang
>



-- 
Thanks,
Neha

Re: Plan of Controlled Shutdown

Posted by Guozhang Wang <wa...@gmail.com>.
Mingjie,

Controlled shutdown has been fully implemented and is regularly exercised
at LinkedIn for version upgrades / rolling bounce tests, etc. As Harsha
said you just need to turn on the config in the brokers, and normal
shutting down command like control-c will trigger the controlled shutdown.

Guozhang

On Wed, Mar 25, 2015 at 12:39 PM, Harsha <ka...@harsha.io> wrote:

> Hi ,
>        you can enable/disable controlled shutdown using
> controlled.shutdown.enable in server.properties. This is by default set to
> “true” in 0.8.2. You can go ahead do a rolling restart  and you don’t need
> ShutdownBroker command which is removed.  Regarding KAFKA-2029 talks about
> improvement to the controlled shutdown especially in case of having high
> partitions per broker which might degrade the performance as per the JIRA.
>
> Thanks,
> Harsha
>
>
> On March 25, 2015 at 11:15:21 AM, Mingjie Lai (mlai@apache.org) wrote:
>
> Hi.
>
> I've been trying to figure out the best way to do kafka broker rolling
> restart, and read the controlled shutdown wiki page:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
>
> However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:
>
> /usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> kafka/admin/ShutdownBroker
> Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class: kafka.admin.ShutdownBroker. Program will
> exit.
>
> Some other commands are there:
>
> /usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
> grep test
>
> test
>
> However, I also see some jira about the bug fixes for ControlledShutdown
> like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
> confused.
>
> What's the plan for this feature in kafka? I still see the value of it, for
> example, we can move some the leader partitions around on purpose for a
> scheduled upgrade or config update, minimize the transition as less as
> possible.
>
> Thanks,
> Mingjie
>



-- 
-- Guozhang

Re: Plan of Controlled Shutdown

Posted by Harsha <ka...@harsha.io>.
Hi ,
       you can enable/disable controlled shutdown using controlled.shutdown.enable in server.properties. This is by default set to “true” in 0.8.2. You can go ahead do a rolling restart  and you don’t need ShutdownBroker command which is removed.  Regarding KAFKA-2029 talks about improvement to the controlled shutdown especially in case of having high partitions per broker which might degrade the performance as per the JIRA.

Thanks,
Harsha


On March 25, 2015 at 11:15:21 AM, Mingjie Lai (mlai@apache.org) wrote:

Hi.  

I've been trying to figure out the best way to do kafka broker rolling  
restart, and read the controlled shutdown wiki page:  

https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown  

However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:  

/usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker  

Exception in thread "main" java.lang.NoClassDefFoundError:  
kafka/admin/ShutdownBroker  
Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker  
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)  
at java.security.AccessController.doPrivileged(Native Method)  
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)  
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)  
Could not find the main class: kafka.admin.ShutdownBroker. Program will  
exit.  

Some other commands are there:  

/usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |  
grep test  

test  

However, I also see some jira about the bug fixes for ControlledShutdown  
like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got  
confused.  

What's the plan for this feature in kafka? I still see the value of it, for  
example, we can move some the leader partitions around on purpose for a  
scheduled upgrade or config update, minimize the transition as less as  
possible.  

Thanks,  
Mingjie