How to move a broker out of rotation?

[Praveen <pr...@gmail.com>]

I have 16 brokers. Now one of the brokers (B-16) got completely messed up
and is sent for repair.

But I can still see some partitions including the B-16 in its replicas,
thereby becoming under-replicated.

Is there a proper way to take broker out of rotation?

Praveen

Re: How to move a broker out of rotation?

[Praveen <pr...@gmail.com>]

Nice. That has some nice set of functionality.

Thanks. I'll take a look.

Praveen

On Thu, Sep 29, 2016 at 4:07 PM, Todd Palino <tp...@gmail.com> wrote:

> There’s not a good answer for this with just the Kafka tools. We opened
> sourced the tool that we use for removing brokers and rebalancing
> partitions in a cluster:
> https://github.com/linkedin/kafka-tools
>
> So when we want to remove a broker (with an ID of 1 in this example) from a
> cluster, we run:
> kafka-assigner -z zookeeper.example.com:2181 -e remove -b 1
>
> That runs a bunch of partition reassignments to move all replicas off that
> broker and distribute them to the other brokers in the cluster.
>
> -Todd
>
>
> On Thu, Sep 29, 2016 at 3:53 PM, Praveen <pr...@gmail.com> wrote:
>
> > I have 16 brokers. Now one of the brokers (B-16) got completely messed up
> > and is sent for repair.
> >
> > But I can still see some partitions including the B-16 in its replicas,
> > thereby becoming under-replicated.
> >
> > Is there a proper way to take broker out of rotation?
> >
> > Praveen
> >
>
>
>
> --
> *Todd Palino*
> Staff Site Reliability Engineer
> Data Infrastructure Streaming
>
>
>
> linkedin.com/in/toddpalino
>

Re: How to move a broker out of rotation?

[Todd Palino <tp...@gmail.com>]

There’s not a good answer for this with just the Kafka tools. We opened
sourced the tool that we use for removing brokers and rebalancing
partitions in a cluster:
https://github.com/linkedin/kafka-tools

So when we want to remove a broker (with an ID of 1 in this example) from a
cluster, we run:
kafka-assigner -z zookeeper.example.com:2181 -e remove -b 1

That runs a bunch of partition reassignments to move all replicas off that
broker and distribute them to the other brokers in the cluster.

-Todd


On Thu, Sep 29, 2016 at 3:53 PM, Praveen <pr...@gmail.com> wrote:

> I have 16 brokers. Now one of the brokers (B-16) got completely messed up
> and is sent for repair.
>
> But I can still see some partitions including the B-16 in its replicas,
> thereby becoming under-replicated.
>
> Is there a proper way to take broker out of rotation?
>
> Praveen
>



-- 
*Todd Palino*
Staff Site Reliability Engineer
Data Infrastructure Streaming



linkedin.com/in/toddpalino