You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Monika Garg <ga...@gmail.com> on 2013/10/25 08:13:22 UTC
Does Kafka stores Producer/Consumer Details
Hi,
I am wondering does metadata of Kafka stores producer/consumer information.
As I start any producer,I provide a broker-list to it.But producer must be
connecting to one of the broker to send data(as per Im gettiing the
things).Similarly I can start more producer by giving a broker-list in the
start command.
So a broker must be storing the information about which producer is
handshaking with it(as a producer will connect to only one broker at any
time).
Same case goes with consumer.
is there any way to find the details about Producer/Consumer from Kafka,if
I am right in explaining the above details?
--
*Moniii*
Re: Does Kafka stores Producer/Consumer Details
Posted by Joel Koshy <jj...@gmail.com>.
>
> I am wondering does metadata of Kafka stores producer/consumer information.
>
> As I start any producer,I provide a broker-list to it.But producer must be
> connecting to one of the broker to send data(as per Im gettiing the
> things).Similarly I can start more producer by giving a broker-list in the
> start command.
>
> So a broker must be storing the information about which producer is
> handshaking with it(as a producer will connect to only one broker at any
> time).
Not really - well, brokers obviously have to maintain these
connections from producers but it maintains no additional per-producer
state.
>
> Same case goes with consumer.
Same for consumers. For high-level consumers, the consumers themselves
maintain state (e.g., registration, partition ownership, offsets) in
zookeeper but there is no per-consumer state on the broker (although
with KAFKA-1000, we will be moving offset management to the broker).
Thanks,
Joel
Re: Does Kafka stores Producer/Consumer Details
Posted by Joel Koshy <jj...@gmail.com>.
> So Joel,can we get the simple details that how many producers's connection
> a broker is maintaining?
I don't think we directly maintain that information but it is readily
available through standard OS-level tools (e.g., netstat).
Thanks,
Joel
>
>
> On Fri, Oct 25, 2013 at 8:25 PM, Jun Rao <ju...@gmail.com> wrote:
>
>> Details of the consumer implementation can be found at
>>
>> http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Thu, Oct 24, 2013 at 11:13 PM, Monika Garg <ga...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> >
>> >
>> > I am wondering does metadata of Kafka stores producer/consumer
>> information.
>> >
>> > As I start any producer,I provide a broker-list to it.But producer must
>> be
>> > connecting to one of the broker to send data(as per Im gettiing the
>> > things).Similarly I can start more producer by giving a broker-list in
>> the
>> > start command.
>> >
>> > So a broker must be storing the information about which producer is
>> > handshaking with it(as a producer will connect to only one broker at any
>> > time).
>> >
>> > Same case goes with consumer.
>> >
>> > is there any way to find the details about Producer/Consumer from
>> Kafka,if
>> > I am right in explaining the above details?
>> >
>> > --
>> > *Moniii*
>> >
>>
>
>
>
> --
> *Moniii*
Re: Does Kafka stores Producer/Consumer Details
Posted by Monika Garg <ga...@gmail.com>.
Thanks Joel and Jun.
So Joel,can we get the simple details that how many producers's connection
a broker is maintaining?
On Fri, Oct 25, 2013 at 8:25 PM, Jun Rao <ju...@gmail.com> wrote:
> Details of the consumer implementation can be found at
>
> http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf
>
> Thanks,
>
> Jun
>
>
> On Thu, Oct 24, 2013 at 11:13 PM, Monika Garg <ga...@gmail.com>
> wrote:
>
> > Hi,
> >
> >
> >
> > I am wondering does metadata of Kafka stores producer/consumer
> information.
> >
> > As I start any producer,I provide a broker-list to it.But producer must
> be
> > connecting to one of the broker to send data(as per Im gettiing the
> > things).Similarly I can start more producer by giving a broker-list in
> the
> > start command.
> >
> > So a broker must be storing the information about which producer is
> > handshaking with it(as a producer will connect to only one broker at any
> > time).
> >
> > Same case goes with consumer.
> >
> > is there any way to find the details about Producer/Consumer from
> Kafka,if
> > I am right in explaining the above details?
> >
> > --
> > *Moniii*
> >
>
--
*Moniii*
Re: Does Kafka stores Producer/Consumer Details
Posted by Jun Rao <ju...@gmail.com>.
Details of the consumer implementation can be found at
http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf
Thanks,
Jun
On Thu, Oct 24, 2013 at 11:13 PM, Monika Garg <ga...@gmail.com> wrote:
> Hi,
>
>
>
> I am wondering does metadata of Kafka stores producer/consumer information.
>
> As I start any producer,I provide a broker-list to it.But producer must be
> connecting to one of the broker to send data(as per Im gettiing the
> things).Similarly I can start more producer by giving a broker-list in the
> start command.
>
> So a broker must be storing the information about which producer is
> handshaking with it(as a producer will connect to only one broker at any
> time).
>
> Same case goes with consumer.
>
> is there any way to find the details about Producer/Consumer from Kafka,if
> I am right in explaining the above details?
>
> --
> *Moniii*
>