You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Milind Parikh <mi...@gmail.com> on 2012/09/28 23:38:09 UTC
MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1
What is the wire format for MULTI-FETCH *RESPONSE* *given
*that the SINGLE-FETCH RESPONSE LOOKS like this:
RESPONSE_HEADER
ResponseLength:32/integer
ErrorCode:16/integer
MESSAGES
(
Length:32/integer
Magic:8/integer
Compression:8/integer
CheckSum:32/integer
Payload/binary
) +
https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't talk
about this.
Thanks
Milind
Re: MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1
Posted by Jun Rao <ju...@gmail.com>.
You can figure out the corresponding topic-partition from the
multiFetchRequest itself. The topic/partition ordering in the response is
the same as that in the request. In 0.8, topic/partition will be included
in the response itself.
Thanks,
Jun
On Fri, Sep 28, 2012 at 3:56 PM, Milind Parikh <mi...@gmail.com>wrote:
> After doing a little wire-snooping of the data, it appears that the
> MULTI-FETCH RESPONSE is EXACTLY the same as the SINGLE-FETCH.
>
> If this is correct, how does one advance to the next offset for each one of
> the topic-partition combo if reading multiple topic-partitions in parallel;
> given that the information about which topic-partition did this message
> originate from is missing in the response?
>
> Is there a JIRA that is getting worked?
>
> Thanks
> Milind
>
>
> On Fri, Sep 28, 2012 at 2:38 PM, Milind Parikh <milindparikh@gmail.com
> >wrote:
>
> > What is the wire format for MULTI-FETCH *RESPONSE* *given
> >
> > *that the SINGLE-FETCH RESPONSE LOOKS like this:
> > RESPONSE_HEADER
> > ResponseLength:32/integer
> > ErrorCode:16/integer
> > MESSAGES
> > (
> > Length:32/integer
> > Magic:8/integer
> > Compression:8/integer
> > CheckSum:32/integer
> > Payload/binary
> > ) +
> >
> >
> > https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't
> > talk about this.
> >
> >
> > Thanks
> > Milind
> >
> >
>
Re: MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1
Posted by Milind Parikh <mi...@gmail.com>.
After doing a little wire-snooping of the data, it appears that the
MULTI-FETCH RESPONSE is EXACTLY the same as the SINGLE-FETCH.
If this is correct, how does one advance to the next offset for each one of
the topic-partition combo if reading multiple topic-partitions in parallel;
given that the information about which topic-partition did this message
originate from is missing in the response?
Is there a JIRA that is getting worked?
Thanks
Milind
On Fri, Sep 28, 2012 at 2:38 PM, Milind Parikh <mi...@gmail.com>wrote:
> What is the wire format for MULTI-FETCH *RESPONSE* *given
>
> *that the SINGLE-FETCH RESPONSE LOOKS like this:
> RESPONSE_HEADER
> ResponseLength:32/integer
> ErrorCode:16/integer
> MESSAGES
> (
> Length:32/integer
> Magic:8/integer
> Compression:8/integer
> CheckSum:32/integer
> Payload/binary
> ) +
>
>
> https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't
> talk about this.
>
>
> Thanks
> Milind
>
>
Re: MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1
Posted by Milind Parikh <mi...@gmail.com>.
After doing a little wire-snooping of the data, it appears that the
MULTI-FETCH RESPONSE is EXACTLY the same as the SINGLE-FETCH.
If this is correct, how does one advance to the next offset for each one of
the topic-partition combo if reading multiple topic-partitions in parallel;
given that the information about which topic-partition did this message
originate from is missing in the response?
Is there a JIRA that is getting worked?
Thanks
Milind
On Fri, Sep 28, 2012 at 2:38 PM, Milind Parikh <mi...@gmail.com>wrote:
> What is the wire format for MULTI-FETCH *RESPONSE* *given
>
> *that the SINGLE-FETCH RESPONSE LOOKS like this:
> RESPONSE_HEADER
> ResponseLength:32/integer
> ErrorCode:16/integer
> MESSAGES
> (
> Length:32/integer
> Magic:8/integer
> Compression:8/integer
> CheckSum:32/integer
> Payload/binary
> ) +
>
>
> https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't
> talk about this.
>
>
> Thanks
> Milind
>
>