You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Kessiler Rodrigues <ke...@callinize.com> on 2016/07/28 20:03:31 UTC

Cluster config

Hey guys, 

I have > 5k topics with 5 partitions each in my cluster today. 

My actual cluster configuration is:

6 brokers - 16 vCPUs, 14.4 GB

Nowadays, I’m having some lags when the consumer tries to do rebalance and sometimes even kafka doesn’t get up.

What would be a best cluster configuration for that?

Re: Cluster config

Posted by Guozhang Wang <wa...@gmail.com>.
Which Kafka version are you using and how many consumers?

Guozhang

On Thu, Jul 28, 2016 at 1:57 PM, Kessiler Rodrigues <ke...@callinize.com>
wrote:

> The replication factor is 4.
>
>
>
> > On Jul 28, 2016, at 5:55 PM, David Garcia <da...@spiceworks.com> wrote:
> >
> > What is your replication for these topics?
> >
> > On 7/28/16, 3:03 PM, "Kessiler Rodrigues" <ke...@callinize.com>
> wrote:
> >
> >    Hey guys,
> >
> >    I have > 5k topics with 5 partitions each in my cluster today.
> >
> >    My actual cluster configuration is:
> >
> >    6 brokers - 16 vCPUs, 14.4 GB
> >
> >    Nowadays, I’m having some lags when the consumer tries to do
> rebalance and sometimes even kafka doesn’t get up.
> >
> >    What would be a best cluster configuration for that?
> >
> >
> >
> >
>
>


-- 
-- Guozhang

Re: Cluster config

Posted by Kessiler Rodrigues <ke...@callinize.com>.
The replication factor is 4.



> On Jul 28, 2016, at 5:55 PM, David Garcia <da...@spiceworks.com> wrote:
> 
> What is your replication for these topics?
> 
> On 7/28/16, 3:03 PM, "Kessiler Rodrigues" <ke...@callinize.com> wrote:
> 
>    Hey guys, 
> 
>    I have > 5k topics with 5 partitions each in my cluster today. 
> 
>    My actual cluster configuration is:
> 
>    6 brokers - 16 vCPUs, 14.4 GB
> 
>    Nowadays, I’m having some lags when the consumer tries to do rebalance and sometimes even kafka doesn’t get up.
> 
>    What would be a best cluster configuration for that?
> 
> 
> 
>

Re: Cluster config

Posted by David Garcia <da...@spiceworks.com>.
What is your replication for these topics?

On 7/28/16, 3:03 PM, "Kessiler Rodrigues" <ke...@callinize.com> wrote:

    Hey guys, 
    
    I have > 5k topics with 5 partitions each in my cluster today. 
    
    My actual cluster configuration is:
    
    6 brokers - 16 vCPUs, 14.4 GB
    
    Nowadays, I’m having some lags when the consumer tries to do rebalance and sometimes even kafka doesn’t get up.
    
    What would be a best cluster configuration for that?