You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Senthil Kumar <se...@gmail.com> on 2017/01/24 17:08:31 UTC
Kafka Multiple Consumer Group for Same Topic
Hi Team , Sorry if the same question asked already in this group !
Say we have topic => ad_events .. I want to read events from ad_events
topic and send it to two different systems... This can be achieved by
creating two Consume Groups..
Example : Consumer Group SYS1 with 10 threads
Consume Group SYS2 with 10 threads
Would like to know having two different Consumer Groups will impact
performance of Kafka Read ?? Also want to see the *Benchmarking Result*(
Numbers ) of Single Topic Read with *One Consumer Group* Vs Single Topic
with *Two/Three Consumer Group*..
Cheers,
Senthil
Re: Kafka Multiple Consumer Group for Same Topic
Posted by Joris Meijer <jo...@dizzit.com>.
Hi Senthil,
You can just try it yourself, using kafka-consumer-perf-test.sh bundled
with Apache Kafka. You can start 2 in parallel with different groups and
see what happens if you tune some parameters.
Joris
On Wed, Jan 25, 2017, 08:38 Senthil Kumar <se...@gmail.com> wrote:
> Thanks Sharninder!
>
> Adding dev group to know if they done some benchmarking test on Single
> Consumer Group Vs Multiple Consumer Grp on Same Topic.
>
> Cheers,
> Senthil
>
>
> On Jan 24, 2017 10:48 PM, "Sharninder Khera" <sh...@gmail.com> wrote:
>
> I don't have benchmarks but multiple consumer groups are possible. For
> Kafka the performance should be similar or close to as having multiple
> consumers using a single group.
>
>
> _____________________________
> From: Senthil Kumar <se...@gmail.com>
> Sent: Tuesday, January 24, 2017 10:38 PM
> Subject: Kafka Multiple Consumer Group for Same Topic
> To: <us...@kafka.apache.org>
> Cc: <se...@apache.org>
>
>
> Hi Team , Sorry if the same question asked already in this group !
>
> Say we have topic => ad_events .. I want to read events from ad_events
> topic and send it to two different systems... This can be achieved by
> creating two Consume Groups..
>
> Example : Consumer Group SYS1 with 10 threads
> Consume Group SYS2 with 10 threads
>
> Would like to know having two different Consumer Groups will impact
> performance of Kafka Read ?? Also want to see the *Benchmarking Result*(
> Numbers ) of Single Topic Read with *One Consumer Group* Vs Single Topic
> with *Two/Three Consumer Group*..
>
>
> Cheers,
> Senthil
>
Re: Kafka Multiple Consumer Group for Same Topic
Posted by Joris Meijer <jo...@dizzit.com>.
Hi Senthil,
You can just try it yourself, using kafka-consumer-perf-test.sh bundled
with Apache Kafka. You can start 2 in parallel with different groups and
see what happens if you tune some parameters.
Joris
On Wed, Jan 25, 2017, 08:38 Senthil Kumar <se...@gmail.com> wrote:
> Thanks Sharninder!
>
> Adding dev group to know if they done some benchmarking test on Single
> Consumer Group Vs Multiple Consumer Grp on Same Topic.
>
> Cheers,
> Senthil
>
>
> On Jan 24, 2017 10:48 PM, "Sharninder Khera" <sh...@gmail.com> wrote:
>
> I don't have benchmarks but multiple consumer groups are possible. For
> Kafka the performance should be similar or close to as having multiple
> consumers using a single group.
>
>
> _____________________________
> From: Senthil Kumar <se...@gmail.com>
> Sent: Tuesday, January 24, 2017 10:38 PM
> Subject: Kafka Multiple Consumer Group for Same Topic
> To: <us...@kafka.apache.org>
> Cc: <se...@apache.org>
>
>
> Hi Team , Sorry if the same question asked already in this group !
>
> Say we have topic => ad_events .. I want to read events from ad_events
> topic and send it to two different systems... This can be achieved by
> creating two Consume Groups..
>
> Example : Consumer Group SYS1 with 10 threads
> Consume Group SYS2 with 10 threads
>
> Would like to know having two different Consumer Groups will impact
> performance of Kafka Read ?? Also want to see the *Benchmarking Result*(
> Numbers ) of Single Topic Read with *One Consumer Group* Vs Single Topic
> with *Two/Three Consumer Group*..
>
>
> Cheers,
> Senthil
>
Re: Kafka Multiple Consumer Group for Same Topic
Posted by Senthil Kumar <se...@gmail.com>.
Thanks Sharninder!
Adding dev group to know if they done some benchmarking test on Single
Consumer Group Vs Multiple Consumer Grp on Same Topic.
Cheers,
Senthil
On Jan 24, 2017 10:48 PM, "Sharninder Khera" <sh...@gmail.com> wrote:
I don't have benchmarks but multiple consumer groups are possible. For
Kafka the performance should be similar or close to as having multiple
consumers using a single group.
_____________________________
From: Senthil Kumar <se...@gmail.com>
Sent: Tuesday, January 24, 2017 10:38 PM
Subject: Kafka Multiple Consumer Group for Same Topic
To: <us...@kafka.apache.org>
Cc: <se...@apache.org>
Hi Team , Sorry if the same question asked already in this group !
Say we have topic => ad_events .. I want to read events from ad_events
topic and send it to two different systems... This can be achieved by
creating two Consume Groups..
Example : Consumer Group SYS1 with 10 threads
Consume Group SYS2 with 10 threads
Would like to know having two different Consumer Groups will impact
performance of Kafka Read ?? Also want to see the *Benchmarking Result*(
Numbers ) of Single Topic Read with *One Consumer Group* Vs Single Topic
with *Two/Three Consumer Group*..
Cheers,
Senthil
Re: Kafka Multiple Consumer Group for Same Topic
Posted by Senthil Kumar <se...@gmail.com>.
Thanks Sharninder!
Adding dev group to know if they done some benchmarking test on Single
Consumer Group Vs Multiple Consumer Grp on Same Topic.
Cheers,
Senthil
On Jan 24, 2017 10:48 PM, "Sharninder Khera" <sh...@gmail.com> wrote:
I don't have benchmarks but multiple consumer groups are possible. For
Kafka the performance should be similar or close to as having multiple
consumers using a single group.
_____________________________
From: Senthil Kumar <se...@gmail.com>
Sent: Tuesday, January 24, 2017 10:38 PM
Subject: Kafka Multiple Consumer Group for Same Topic
To: <us...@kafka.apache.org>
Cc: <se...@apache.org>
Hi Team , Sorry if the same question asked already in this group !
Say we have topic => ad_events .. I want to read events from ad_events
topic and send it to two different systems... This can be achieved by
creating two Consume Groups..
Example : Consumer Group SYS1 with 10 threads
Consume Group SYS2 with 10 threads
Would like to know having two different Consumer Groups will impact
performance of Kafka Read ?? Also want to see the *Benchmarking Result*(
Numbers ) of Single Topic Read with *One Consumer Group* Vs Single Topic
with *Two/Three Consumer Group*..
Cheers,
Senthil
Re: Kafka Multiple Consumer Group for Same Topic
Posted by Sharninder Khera <sh...@gmail.com>.
I don't have benchmarks but multiple consumer groups are possible. For Kafka the performance should be similar or close to as having multiple consumers using a single group.
_____________________________
From: Senthil Kumar <se...@gmail.com>
Sent: Tuesday, January 24, 2017 10:38 PM
Subject: Kafka Multiple Consumer Group for Same Topic
To: <us...@kafka.apache.org>
Cc: <se...@apache.org>
Hi Team , Sorry if the same question asked already in this group !
Say we have topic => ad_events .. I want to read events from ad_events
topic and send it to two different systems... This can be achieved by
creating two Consume Groups..
Example : Consumer Group SYS1 with 10 threads
Consume Group SYS2 with 10 threads
Would like to know having two different Consumer Groups will impact
performance of Kafka Read ?? Also want to see the *Benchmarking Result*(
Numbers ) of Single Topic Read with *One Consumer Group* Vs Single Topic
with *Two/Three Consumer Group*..
Cheers,
Senthil