How to enable RACK awareness on Already Running Kafka Cluster

[Ashutosh singh <ge...@gmail.com>]

Greetings,

We have 8 nodes Brokers setup on AWS in 2 availability zones with 2
replication. Our plan is to add one more node and distribute 3 nodes in
each AZ and change replication factor to 3.

In order to replicate data in each AZ we need to enable Rack awareness. Can
someone guide how can I achieve this ? I can bring down only one broker and
if I change settings in one broker i.e. (put broker.rack=rackid) and
restart the Kafka then what would be impact ? how it will handle with the
requests because other brokers have not these settings yet.

Also, we have 1000+ topics in our cluster , do we need to manually reassign
partitions for all topics ?

I have searched everywhere but couldn't find any place where it says if can
do it while kafka cluster is in operation.

I really appreciate if someone can help on this.


-- 
Regard
Ashu

Re: How to enable RACK awareness on Already Running Kafka Cluster

[Ashutosh singh <ge...@gmail.com>]

Thanks a lot Peter.   I searched this line "It’s possible to enable
rack-awareness in a rolling manner"  everywhere over the internet but
coudn't find it.   Thank you for your help. Have a real nice day.

On Thu, Sep 19, 2019 at 10:43 PM Peter Bukowinski <pm...@gmail.com> wrote:

> Hi Ashu,
>
> It’s possible to enable rack-awareness in a rolling manner. Kafka will
> never automatically move existing partitions, unless you tell it to or have
> a separate tool (e.g. Cruise Control) that does it for you. Rack-awareness
> comes into play when topics are initially created and partitions are
> distributed around the cluster.
>
> After you’ve set the broker.rack property for each broker and have
> restarted them, you will need to do a manual rebalance to add a third
> replica and properly distribute your replicas between the availability
> zones.
>
> —
> Peter
>
> > On Sep 19, 2019, at 12:17 AM, Ashutosh singh <ge...@gmail.com> wrote:
> >
> > Greetings,
> >
> > We have 8 nodes Brokers setup on AWS in 2 availability zones with 2
> > replication. Our plan is to add one more node and distribute 3 nodes in
> > each AZ and change replication factor to 3.
> >
> > In order to replicate data in each AZ we need to enable Rack awareness.
> Can
> > someone guide how can I achieve this ? I can bring down only one broker
> and
> > if I change settings in one broker i.e. (put broker.rack=rackid) and
> > restart the Kafka then what would be impact ? how it will handle with the
> > requests because other brokers have not these settings yet.
> >
> > Also, we have 1000+ topics in our cluster , do we need to manually
> reassign
> > partitions for all topics ?
> >
> > I have searched everywhere but couldn't find any place where it says if
> can
> > do it while kafka cluster is in operation.
> >
> > I really appreciate if someone can help on this.
> >
> >
> > --
> > Regard
> > Ashu
>
>

-- 
Thanx & Regard
Ashutosh Singh
08151945559

Re: How to enable RACK awareness on Already Running Kafka Cluster

[Peter Bukowinski <pm...@gmail.com>]

Hi Ashu,

It’s possible to enable rack-awareness in a rolling manner. Kafka will never automatically move existing partitions, unless you tell it to or have a separate tool (e.g. Cruise Control) that does it for you. Rack-awareness comes into play when topics are initially created and partitions are distributed around the cluster.

After you’ve set the broker.rack property for each broker and have restarted them, you will need to do a manual rebalance to add a third replica and properly distribute your replicas between the availability zones.

—
Peter

> On Sep 19, 2019, at 12:17 AM, Ashutosh singh <ge...@gmail.com> wrote:
> 
> Greetings,
> 
> We have 8 nodes Brokers setup on AWS in 2 availability zones with 2
> replication. Our plan is to add one more node and distribute 3 nodes in
> each AZ and change replication factor to 3.
> 
> In order to replicate data in each AZ we need to enable Rack awareness. Can
> someone guide how can I achieve this ? I can bring down only one broker and
> if I change settings in one broker i.e. (put broker.rack=rackid) and
> restart the Kafka then what would be impact ? how it will handle with the
> requests because other brokers have not these settings yet.
> 
> Also, we have 1000+ topics in our cluster , do we need to manually reassign
> partitions for all topics ?
> 
> I have searched everywhere but couldn't find any place where it says if can
> do it while kafka cluster is in operation.
> 
> I really appreciate if someone can help on this.
> 
> 
> -- 
> Regard
> Ashu