Mapping a consumer in a consumer group to a partition in a topic

[Spandan Harithas Karamchedu <sp...@gmail.com>]

Hi,

We created a topic with 3 partitions and a replication factor of 3. We are
able to implement a consumer to get the data from a specific partition in a
topic but we are struck in implementing a consumer within a specified
consumer group to be mapped to single partition of a topic and get the data
from the single partition of the topic. Can you let us how we can configure
a consumer to a partition within consumer group?



We wanted to develop a kafka cluster similar to below one as specified in
kafka docs. We are using kafka 0.8.1.




Thanks

Spandan

Re: Mapping a consumer in a consumer group to a partition in a topic

[Sharninder <sh...@gmail.com>]

As far as I know, with a consumer group implementation you cannot pin consumers to partitions. That logic is taken care of by the high level API on its own. 

> On 23-Sep-2015, at 6:38 AM, Spandan Harithas Karamchedu <sp...@gmail.com> wrote:
> 
> Hi,
> 
> We created a topic with 3 partitions and a replication factor of 3. We are
> able to implement a consumer to get the data from a specific partition in a
> topic but we are struck in implementing a consumer within a specified
> consumer group to be mapped to single partition of a topic and get the data
> from the single partition of the topic. Can you let us how we can configure
> a consumer to a partition within consumer group?
> 
> 
> 
> We wanted to develop a kafka cluster similar to below one as specified in
> kafka docs. We are using kafka 0.8.1.
> 
> 
> 
> 
> Thanks
> 
> Spandan

Re: Mapping a consumer in a consumer group to a partition in a topic

[Sharninder <sh...@gmail.com>]

As far as I know, with a consumer group implementation you cannot pin consumers to partitions. That logic is taken care of by the high level API on its own. 

> On 23-Sep-2015, at 6:38 AM, Spandan Harithas Karamchedu <sp...@gmail.com> wrote:
> 
> Hi,
> 
> We created a topic with 3 partitions and a replication factor of 3. We are
> able to implement a consumer to get the data from a specific partition in a
> topic but we are struck in implementing a consumer within a specified
> consumer group to be mapped to single partition of a topic and get the data
> from the single partition of the topic. Can you let us how we can configure
> a consumer to a partition within consumer group?
> 
> 
> 
> We wanted to develop a kafka cluster similar to below one as specified in
> kafka docs. We are using kafka 0.8.1.
> 
> 
> 
> 
> Thanks
> 
> Spandan

Re: Mapping a consumer in a consumer group to a partition in a topic

[Gwen Shapira <gw...@confluent.io>]

Unfortunately, in order to get a specific partition, you will need to use
the simple consumer API, which does not have consumer groups.

see here for details:
https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example

On Tue, Sep 22, 2015 at 6:08 PM, Spandan Harithas Karamchedu <
spandanharithas@gmail.com> wrote:

> Hi,
>
> We created a topic with 3 partitions and a replication factor of 3. We are
> able to implement a consumer to get the data from a specific partition in a
> topic but we are struck in implementing a consumer within a specified
> consumer group to be mapped to single partition of a topic and get the data
> from the single partition of the topic. Can you let us how we can configure
> a consumer to a partition within consumer group?
>
>
>
> We wanted to develop a kafka cluster similar to below one as specified in
> kafka docs. We are using kafka 0.8.1.
>
>
>
>
> Thanks
>
> Spandan
>