Number of concurrent consumers per data node

[Shane MacPhillamy <sh...@blinkmobile.com.au>]

Hi

I’m just coming up to speed with Kafka. Some beginner questions, may be point me to where I can find the answers please:

1. In a Kafka cluster what determines the maximum number of concurrent consumers that may be connected to a data node?

2. What is a/the scale out strategy for a Kafka cluster when data nodes run out of sockets for a consumer to connect to?

Thanks.

Cheers, Shane

Re: Number of concurrent consumers per data node

[Ewen Cheslack-Postava <ew...@confluent.io>]

On Wed, Feb 3, 2016 at 3:57 PM, Shane MacPhillamy <sh...@blinkmobile.com.au>
wrote:

> Hi
>
> I’m just coming up to speed with Kafka. Some beginner questions, may be
> point me to where I can find the answers please:
>
> 1. In a Kafka cluster what determines the maximum number of concurrent
> consumers that may be connected to a data node?
>

There's not limit imposed by Kafka. Various resource limits (sockets,
bandwidth, etc) will be the limiting factors.


>
> 2. What is a/the scale out strategy for a Kafka cluster when data nodes
> run out of sockets for a consumer to connect to?
>

Currently, the expectation is that you read from a partition leader. The
scale out strategy for a *topic* is to have sufficient partitions, where
partition leaders are spread across many brokers, to support the desired
load.

-Ewen


>
> Thanks.
>
> Cheers, Shane




-- 
Thanks,
Ewen