bandwidth usage issue

["Yu, Libo " <li...@citi.com>]

Hi team,

Here is a usage case: Assume each host in a kafka cluster a gigabit network adaptor.
And the incoming traffic is 0.8gbps and at one point all the traffic goes to one host.
The remaining bandwidth is not enough for the followers to replicate messages from
this leader.

To make sure no broker will be dropped out of ISR, the intra-cluster bandwidth must
be much larger than incoming/outgoing bandwidth. Otherwise, use replication factor 1.
Is my understanding correct? Thanks.


Regards,

Libo

Re: bandwidth usage issue

[Neha Narkhede <ne...@gmail.com>]

This is a capacity planning issue. I think the right thing to do here is to
expand the cluster and use the partition reassignment tool to move some
partitions over to the new brokers to evenly spread out the load.

Thanks,
Neha


On Tue, Oct 1, 2013 at 8:53 AM, Yu, Libo <li...@citi.com> wrote:

> Hi team,
>
> Here is a usage case: Assume each host in a kafka cluster a gigabit
> network adaptor.
> And the incoming traffic is 0.8gbps and at one point all the traffic goes
> to one host.
> The remaining bandwidth is not enough for the followers to replicate
> messages from
> this leader.
>
> To make sure no broker will be dropped out of ISR, the intra-cluster
> bandwidth must
> be much larger than incoming/outgoing bandwidth. Otherwise, use
> replication factor 1.
> Is my understanding correct? Thanks.
>
>
> Regards,
>
> Libo
>
>