kafka producer and client use same groupid

["Kudumula, Surender" <su...@hpe.com>]

Hi all
Is there anyway we can ensure in 0.8 that kafka remote producer and remote consumer work on the same groupId as my java consumer cannot consume messages from remote producer. Thanks

Re: kafka producer and client use same groupid

[Muqtafi Akhmad <mu...@traveloka.com>]

Hello Surender,

do you use the high level consumer, low level consumer, or using connector
provided by Spark / Hadoop?

On Wed, Nov 25, 2015 at 1:29 AM, Kudumula, Surender <
surender.kudumula@hpe.com> wrote:

> Hi Prabhjot
> Thanks for your quick response. I never get any response to my queries:( .
> Anyway my issue here is I have my kafka running as part of hdp cluster. I
> have my producer in the same cluster producing to a topic in kafka and
> consumer which is on another node trying to consume the messages from the
> same topic. The issue I have here is when I run the command line consumer
> it consumes all the messages whereas my java client consumer doesn’t
> consume a single message. Any ideas please?
>
>
> -----Original Message-----
> From: Prabhjot Bharaj [mailto:prabhbharaj@gmail.com]
> Sent: 24 November 2015 18:24
> To: dev@kafka.apache.org
> Cc: users@kafka.apache.org
> Subject: Re: kafka producer and client use same groupid
>
> Hi Surender,
>
> Please elaborate on your design
> Consumers don't talk to producers directly, Kafka is a brokered system,
> and Kafka sits between producers and consumers Also, consumers consume from
> partitions of a topic and producers write to partitions in a topic These
> partitions and the logical abstraction -topic-reside on Kafka and zookeeper
> respectively
>
> Thanks,
> Prabhjot
> On Nov 24, 2015 11:38 PM, "Kudumula, Surender" <su...@hpe.com>
> wrote:
>
> > Hi all
> > Is there anyway we can ensure in 0.8 that kafka remote producer and
> > remote consumer work on the same groupId as my java consumer cannot
> > consume messages from remote producer. Thanks
> >
> >
> >
> >
>



-- 
Muqtafi Akhmad
Software Engineer
Traveloka

RE: kafka producer and client use same groupid

["Kudumula, Surender" <su...@hpe.com>]

Hi Prabhjot
Thanks for your quick response. I never get any response to my queries:( . Anyway my issue here is I have my kafka running as part of hdp cluster. I have my producer in the same cluster producing to a topic in kafka and consumer which is on another node trying to consume the messages from the same topic. The issue I have here is when I run the command line consumer it consumes all the messages whereas my java client consumer doesn’t consume a single message. Any ideas please?


-----Original Message-----
From: Prabhjot Bharaj [mailto:prabhbharaj@gmail.com] 
Sent: 24 November 2015 18:24
To: dev@kafka.apache.org
Cc: users@kafka.apache.org
Subject: Re: kafka producer and client use same groupid

Hi Surender,

Please elaborate on your design
Consumers don't talk to producers directly, Kafka is a brokered system, and Kafka sits between producers and consumers Also, consumers consume from partitions of a topic and producers write to partitions in a topic These partitions and the logical abstraction -topic-reside on Kafka and zookeeper respectively

Thanks,
Prabhjot
On Nov 24, 2015 11:38 PM, "Kudumula, Surender" <su...@hpe.com>
wrote:

> Hi all
> Is there anyway we can ensure in 0.8 that kafka remote producer and 
> remote consumer work on the same groupId as my java consumer cannot 
> consume messages from remote producer. Thanks
>
>
>
>

Re: kafka producer and client use same groupid

[Prabhjot Bharaj <pr...@gmail.com>]

Hi Surender,

Please elaborate on your design
Consumers don't talk to producers directly, Kafka is a brokered system, and
Kafka sits between producers and consumers
Also, consumers consume from partitions of a topic and producers write to
partitions in a topic
These partitions and the logical abstraction -topic-reside on Kafka and
zookeeper respectively

Thanks,
Prabhjot
On Nov 24, 2015 11:38 PM, "Kudumula, Surender" <su...@hpe.com>
wrote:

> Hi all
> Is there anyway we can ensure in 0.8 that kafka remote producer and remote
> consumer work on the same groupId as my java consumer cannot consume
> messages from remote producer. Thanks
>
>
>
>

Re: kafka producer and client use same groupid

[Prabhjot Bharaj <pr...@gmail.com>]

Hi Surender,

Please elaborate on your design
Consumers don't talk to producers directly, Kafka is a brokered system, and
Kafka sits between producers and consumers
Also, consumers consume from partitions of a topic and producers write to
partitions in a topic
These partitions and the logical abstraction -topic-reside on Kafka and
zookeeper respectively

Thanks,
Prabhjot
On Nov 24, 2015 11:38 PM, "Kudumula, Surender" <su...@hpe.com>
wrote:

> Hi all
> Is there anyway we can ensure in 0.8 that kafka remote producer and remote
> consumer work on the same groupId as my java consumer cannot consume
> messages from remote producer. Thanks
>
>
>
>

Re: kafka producer and client use same groupid

[Gaurav Agarwal <ga...@gmail.com>]

Can u check couple of things
1. Message size that u are sending ttry sending small message
2.check which encoder defaultencoder or string encoder u are using to
consume the message , are u serializing the message while sending normal
stream.
3.u create.any partition of topic just create only topic no partitions
On Nov 24, 2015 11:38 PM, "Kudumula, Surender" <su...@hpe.com>
wrote:

> Hi all
> Is there anyway we can ensure in 0.8 that kafka remote producer and remote
> consumer work on the same groupId as my java consumer cannot consume
> messages from remote producer. Thanks
>
>
>
>