You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Milind Parikh <mi...@gmail.com> on 2012/09/28 23:38:09 UTC

MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1

What is the wire format for MULTI-FETCH *RESPONSE* *given

   *that the SINGLE-FETCH RESPONSE LOOKS like this:
                    RESPONSE_HEADER
                                  ResponseLength:32/integer
                                  ErrorCode:16/integer
                   MESSAGES
                             (
                                  Length:32/integer
                                  Magic:8/integer
                                  Compression:8/integer
                                  CheckSum:32/integer
                                  Payload/binary
                             ) +


https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't talk
about this.


Thanks
Milind

Re: MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1

Posted by Jun Rao <ju...@gmail.com>.
You can figure out the corresponding topic-partition from the
multiFetchRequest itself. The topic/partition ordering in the response is
the same as that in the request. In 0.8, topic/partition will be included
in the response itself.

Thanks,

Jun

On Fri, Sep 28, 2012 at 3:56 PM, Milind Parikh <mi...@gmail.com>wrote:

> After doing a little wire-snooping of the data, it appears that the
> MULTI-FETCH RESPONSE is EXACTLY the same as the SINGLE-FETCH.
>
> If this is correct, how does one advance to the next offset for each one of
> the topic-partition combo if reading multiple topic-partitions in parallel;
> given that the information about which topic-partition did this message
> originate from is missing in the response?
>
> Is there a JIRA that is getting worked?
>
> Thanks
> Milind
>
>
> On Fri, Sep 28, 2012 at 2:38 PM, Milind Parikh <milindparikh@gmail.com
> >wrote:
>
> > What is the wire format for MULTI-FETCH *RESPONSE* *given
> >
> >    *that the SINGLE-FETCH RESPONSE LOOKS like this:
> >                     RESPONSE_HEADER
> >                                   ResponseLength:32/integer
> >                                   ErrorCode:16/integer
> >                    MESSAGES
> >                              (
> >                                   Length:32/integer
> >                                   Magic:8/integer
> >                                   Compression:8/integer
> >                                   CheckSum:32/integer
> >                                   Payload/binary
> >                              ) +
> >
> >
> > https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't
> > talk about this.
> >
> >
> > Thanks
> > Milind
> >
> >
>

Re: MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1

Posted by Milind Parikh <mi...@gmail.com>.
After doing a little wire-snooping of the data, it appears that the
MULTI-FETCH RESPONSE is EXACTLY the same as the SINGLE-FETCH.

If this is correct, how does one advance to the next offset for each one of
the topic-partition combo if reading multiple topic-partitions in parallel;
given that the information about which topic-partition did this message
originate from is missing in the response?

Is there a JIRA that is getting worked?

Thanks
Milind


On Fri, Sep 28, 2012 at 2:38 PM, Milind Parikh <mi...@gmail.com>wrote:

> What is the wire format for MULTI-FETCH *RESPONSE* *given
>
>    *that the SINGLE-FETCH RESPONSE LOOKS like this:
>                     RESPONSE_HEADER
>                                   ResponseLength:32/integer
>                                   ErrorCode:16/integer
>                    MESSAGES
>                              (
>                                   Length:32/integer
>                                   Magic:8/integer
>                                   Compression:8/integer
>                                   CheckSum:32/integer
>                                   Payload/binary
>                              ) +
>
>
> https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't
> talk about this.
>
>
> Thanks
> Milind
>
>

Re: MULTI FETCH RESPONSE Wire Format for Kafka 0.7.1

Posted by Milind Parikh <mi...@gmail.com>.
After doing a little wire-snooping of the data, it appears that the
MULTI-FETCH RESPONSE is EXACTLY the same as the SINGLE-FETCH.

If this is correct, how does one advance to the next offset for each one of
the topic-partition combo if reading multiple topic-partitions in parallel;
given that the information about which topic-partition did this message
originate from is missing in the response?

Is there a JIRA that is getting worked?

Thanks
Milind


On Fri, Sep 28, 2012 at 2:38 PM, Milind Parikh <mi...@gmail.com>wrote:

> What is the wire format for MULTI-FETCH *RESPONSE* *given
>
>    *that the SINGLE-FETCH RESPONSE LOOKS like this:
>                     RESPONSE_HEADER
>                                   ResponseLength:32/integer
>                                   ErrorCode:16/integer
>                    MESSAGES
>                              (
>                                   Length:32/integer
>                                   Magic:8/integer
>                                   Compression:8/integer
>                                   CheckSum:32/integer
>                                   Payload/binary
>                              ) +
>
>
> https://cwiki.apache.org/KAFKA/writing-a-driver-for-kafka.html doesn't
> talk about this.
>
>
> Thanks
> Milind
>
>