You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Mingjie Lai <ml...@apache.org> on 2015/03/25 19:15:17 UTC
Plan of Controlled Shutdown
Hi.
I've been trying to figure out the best way to do kafka broker rolling
restart, and read the controlled shutdown wiki page:
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:
/usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker
Exception in thread "main" java.lang.NoClassDefFoundError:
kafka/admin/ShutdownBroker
Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: kafka.admin.ShutdownBroker. Program will
exit.
Some other commands are there:
/usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
grep test
test
However, I also see some jira about the bug fixes for ControlledShutdown
like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
confused.
What's the plan for this feature in kafka? I still see the value of it, for
example, we can move some the leader partitions around on purpose for a
scheduled upgrade or config update, minimize the transition as less as
possible.
Thanks,
Mingjie
Re: Plan of Controlled Shutdown
Posted by Neha Narkhede <ne...@confluent.io>.
Removed the mention of the controlled shutdown tool from the wiki since we
don't support it anymore -
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
On Wed, Mar 25, 2015 at 3:07 PM, Guozhang Wang <wa...@gmail.com> wrote:
> Mingjie,
>
> Controlled shutdown has been fully implemented and is regularly exercised
> at LinkedIn for version upgrades / rolling bounce tests, etc. As Harsha
> said you just need to turn on the config in the brokers, and normal
> shutting down command like control-c will trigger the controlled shutdown.
>
> Guozhang
>
> On Wed, Mar 25, 2015 at 12:39 PM, Harsha <ka...@harsha.io> wrote:
>
> > Hi ,
> > you can enable/disable controlled shutdown using
> > controlled.shutdown.enable in server.properties. This is by default set
> to
> > “true” in 0.8.2. You can go ahead do a rolling restart and you don’t
> need
> > ShutdownBroker command which is removed. Regarding KAFKA-2029 talks
> about
> > improvement to the controlled shutdown especially in case of having high
> > partitions per broker which might degrade the performance as per the
> JIRA.
> >
> > Thanks,
> > Harsha
> >
> >
> > On March 25, 2015 at 11:15:21 AM, Mingjie Lai (mlai@apache.org) wrote:
> >
> > Hi.
> >
> > I've been trying to figure out the best way to do kafka broker rolling
> > restart, and read the controlled shutdown wiki page:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
> >
> > However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:
> >
> > /usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > kafka/admin/ShutdownBroker
> > Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> > Could not find the main class: kafka.admin.ShutdownBroker. Program will
> > exit.
> >
> > Some other commands are there:
> >
> > /usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
> > grep test
> >
> > test
> >
> > However, I also see some jira about the bug fixes for ControlledShutdown
> > like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
> > confused.
> >
> > What's the plan for this feature in kafka? I still see the value of it,
> for
> > example, we can move some the leader partitions around on purpose for a
> > scheduled upgrade or config update, minimize the transition as less as
> > possible.
> >
> > Thanks,
> > Mingjie
> >
>
>
>
> --
> -- Guozhang
>
--
Thanks,
Neha
Re: Plan of Controlled Shutdown
Posted by Guozhang Wang <wa...@gmail.com>.
Mingjie,
Controlled shutdown has been fully implemented and is regularly exercised
at LinkedIn for version upgrades / rolling bounce tests, etc. As Harsha
said you just need to turn on the config in the brokers, and normal
shutting down command like control-c will trigger the controlled shutdown.
Guozhang
On Wed, Mar 25, 2015 at 12:39 PM, Harsha <ka...@harsha.io> wrote:
> Hi ,
> you can enable/disable controlled shutdown using
> controlled.shutdown.enable in server.properties. This is by default set to
> “true” in 0.8.2. You can go ahead do a rolling restart and you don’t need
> ShutdownBroker command which is removed. Regarding KAFKA-2029 talks about
> improvement to the controlled shutdown especially in case of having high
> partitions per broker which might degrade the performance as per the JIRA.
>
> Thanks,
> Harsha
>
>
> On March 25, 2015 at 11:15:21 AM, Mingjie Lai (mlai@apache.org) wrote:
>
> Hi.
>
> I've been trying to figure out the best way to do kafka broker rolling
> restart, and read the controlled shutdown wiki page:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
>
> However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:
>
> /usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> kafka/admin/ShutdownBroker
> Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class: kafka.admin.ShutdownBroker. Program will
> exit.
>
> Some other commands are there:
>
> /usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
> grep test
>
> test
>
> However, I also see some jira about the bug fixes for ControlledShutdown
> like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
> confused.
>
> What's the plan for this feature in kafka? I still see the value of it, for
> example, we can move some the leader partitions around on purpose for a
> scheduled upgrade or config update, minimize the transition as less as
> possible.
>
> Thanks,
> Mingjie
>
--
-- Guozhang
Re: Plan of Controlled Shutdown
Posted by Harsha <ka...@harsha.io>.
Hi ,
you can enable/disable controlled shutdown using controlled.shutdown.enable in server.properties. This is by default set to “true” in 0.8.2. You can go ahead do a rolling restart and you don’t need ShutdownBroker command which is removed. Regarding KAFKA-2029 talks about improvement to the controlled shutdown especially in case of having high partitions per broker which might degrade the performance as per the JIRA.
Thanks,
Harsha
On March 25, 2015 at 11:15:21 AM, Mingjie Lai (mlai@apache.org) wrote:
Hi.
I've been trying to figure out the best way to do kafka broker rolling
restart, and read the controlled shutdown wiki page:
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-1.ControlledShutdown
However I cannot find this kafka.admin.ShutdownBroker class in 0.8.2:
/usr/lib/kafka $ bin/kafka-run-class.sh kafka.admin.ShutdownBroker
Exception in thread "main" java.lang.NoClassDefFoundError:
kafka/admin/ShutdownBroker
Caused by: java.lang.ClassNotFoundException: kafka.admin.ShutdownBroker
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: kafka.admin.ShutdownBroker. Program will
exit.
Some other commands are there:
/usr/lib/kafka $ bin/kafka-topics.sh --list --zookeeper zk1:2181/kafka |
grep test
test
However, I also see some jira about the bug fixes for ControlledShutdown
like https://issues.apache.org/jira/browse/KAFKA-2029. So I kind of got
confused.
What's the plan for this feature in kafka? I still see the value of it, for
example, we can move some the leader partitions around on purpose for a
scheduled upgrade or config update, minimize the transition as less as
possible.
Thanks,
Mingjie