You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Andrew Otto <ao...@wikimedia.org> on 2015/07/30 23:37:38 UTC
Decomissioning a broker
I’m sure this has been asked before, but I can’t seem to find the answer.
I’m planning a Kafka cluster expansion and upgrade to 0.8.2.1. In doing so, I will be decommissioning a broker. I plan to remove this broker fully from the cluster, and then reinstall it and use it for a different purpose.
I understand how to use the reassign-partitions tool to generate new partition assignments and to move partitions around so that the target broker no longer has any active replicas. Once that is done, is there anything special that needs to happen? I can shutdown the broker, but as far as I know that broker will still be registered in Zookeeper. Should I just delete the znode for that broker once it has been shut down?
Thanks!
-Andrew Otto
Re: Decomissioning a broker
Posted by Grant Henke <gh...@cloudera.com>.
Thats correct. Thanks for catching that.
On Tue, Aug 4, 2015 at 3:27 PM, Andrew Otto <ao...@wikimedia.org> wrote:
> Thanks!
>
> > In fact if you use a "Controlled Shutdown" migrating the replicas and
> > leaders should happen for you as well.
>
> Just to clarify, controlled shutdown will only move the leaders to other
> replicas, right? It won’t actually migrate any replicas elsewhere.
>
> -Ao
>
>
> > On Aug 4, 2015, at 13:00, Grant Henke <gh...@cloudera.com> wrote:
> >
> > The broker will actually unregister itself from zookeeper. The brokers id
> > path uses ephemeral nodes so they are automatically destroyed on
> shutdown.
> > In fact if you use a "Controlled Shutdown" migrating the replicas and
> > leaders should happen for you as well. Though, manual reassignment may be
> > preferred in your case.
> >
> > Here is some extra information on controlled shutdowns:
> > http://kafka.apache.org/documentation.html#basic_ops_restarting
> >
> > Thanks,
> > Grant
> >
> > On Thu, Jul 30, 2015 at 4:37 PM, Andrew Otto <ao...@wikimedia.org>
> wrote:
> >
> >> I’m sure this has been asked before, but I can’t seem to find the
> answer.
> >>
> >> I’m planning a Kafka cluster expansion and upgrade to 0.8.2.1. In doing
> >> so, I will be decommissioning a broker. I plan to remove this broker
> fully
> >> from the cluster, and then reinstall it and use it for a different
> purpose.
> >>
> >> I understand how to use the reassign-partitions tool to generate new
> >> partition assignments and to move partitions around so that the target
> >> broker no longer has any active replicas. Once that is done, is there
> >> anything special that needs to happen? I can shutdown the broker, but
> as
> >> far as I know that broker will still be registered in Zookeeper.
> Should I
> >> just delete the znode for that broker once it has been shut down?
> >>
> >> Thanks!
> >> -Andrew Otto
> >>
> >>
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: Decomissioning a broker
Posted by Andrew Otto <ao...@wikimedia.org>.
Thanks!
> In fact if you use a "Controlled Shutdown" migrating the replicas and
> leaders should happen for you as well.
Just to clarify, controlled shutdown will only move the leaders to other replicas, right? It won’t actually migrate any replicas elsewhere.
-Ao
> On Aug 4, 2015, at 13:00, Grant Henke <gh...@cloudera.com> wrote:
>
> The broker will actually unregister itself from zookeeper. The brokers id
> path uses ephemeral nodes so they are automatically destroyed on shutdown.
> In fact if you use a "Controlled Shutdown" migrating the replicas and
> leaders should happen for you as well. Though, manual reassignment may be
> preferred in your case.
>
> Here is some extra information on controlled shutdowns:
> http://kafka.apache.org/documentation.html#basic_ops_restarting
>
> Thanks,
> Grant
>
> On Thu, Jul 30, 2015 at 4:37 PM, Andrew Otto <ao...@wikimedia.org> wrote:
>
>> I’m sure this has been asked before, but I can’t seem to find the answer.
>>
>> I’m planning a Kafka cluster expansion and upgrade to 0.8.2.1. In doing
>> so, I will be decommissioning a broker. I plan to remove this broker fully
>> from the cluster, and then reinstall it and use it for a different purpose.
>>
>> I understand how to use the reassign-partitions tool to generate new
>> partition assignments and to move partitions around so that the target
>> broker no longer has any active replicas. Once that is done, is there
>> anything special that needs to happen? I can shutdown the broker, but as
>> far as I know that broker will still be registered in Zookeeper. Should I
>> just delete the znode for that broker once it has been shut down?
>>
>> Thanks!
>> -Andrew Otto
>>
>>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: Decomissioning a broker
Posted by Grant Henke <gh...@cloudera.com>.
The broker will actually unregister itself from zookeeper. The brokers id
path uses ephemeral nodes so they are automatically destroyed on shutdown.
In fact if you use a "Controlled Shutdown" migrating the replicas and
leaders should happen for you as well. Though, manual reassignment may be
preferred in your case.
Here is some extra information on controlled shutdowns:
http://kafka.apache.org/documentation.html#basic_ops_restarting
Thanks,
Grant
On Thu, Jul 30, 2015 at 4:37 PM, Andrew Otto <ao...@wikimedia.org> wrote:
> I’m sure this has been asked before, but I can’t seem to find the answer.
>
> I’m planning a Kafka cluster expansion and upgrade to 0.8.2.1. In doing
> so, I will be decommissioning a broker. I plan to remove this broker fully
> from the cluster, and then reinstall it and use it for a different purpose.
>
> I understand how to use the reassign-partitions tool to generate new
> partition assignments and to move partitions around so that the target
> broker no longer has any active replicas. Once that is done, is there
> anything special that needs to happen? I can shutdown the broker, but as
> far as I know that broker will still be registered in Zookeeper. Should I
> just delete the znode for that broker once it has been shut down?
>
> Thanks!
> -Andrew Otto
>
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke