You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Dave Peterson <ds...@tagged.com> on 2014/07/23 23:09:40 UTC
question about message sets and produce requests/responses
Hello,
I have a question about produce requests and responses.
Suppose I create a produce request consisting of a single topic T
which contains two message sets S1 and S2, with S1 preceding
S2. In other words, the request looks like this:
( request header ( T ( S1, S2 ) ) )
Is it possible that I may get a response in which the order of
the ACKs for the individual message sets is reversed? In other
words, a response that looks like this:
( response header ( T ( ack for S2, ack for S1 ) ) )
I know that if a request contains multiple topics, then the topics
may arrive out of order in the response. So I was wondering if
ACKs for individual message sets within a topic can likewise
arrive out of order in the response.
Thanks,
Dave
Re: question about message sets and produce requests/responses
Posted by Joel Koshy <jj...@gmail.com>.
Also, to be clear, by ack I mean an error code as described here:
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ProduceResponse
On Wed, Jul 23, 2014 at 04:33:29PM -0700, Joel Koshy wrote:
> Not sure if I followed the question correctly, but S1 and S2 would be
> message-sets to different partitions of T - the producer request
> object contains a map of topic-partition -> messageset). So each
> partition would get its own ack. The response object is just a map of
> partition -> response status so the order of acks in the response is
> the response status map's order.
>
> Joel
>
> On Wed, Jul 23, 2014 at 02:09:40PM -0700, Dave Peterson wrote:
> > Hello,
> >
> > I have a question about produce requests and responses.
> > Suppose I create a produce request consisting of a single topic T
> > which contains two message sets S1 and S2, with S1 preceding
> > S2. In other words, the request looks like this:
> >
> > ( request header ( T ( S1, S2 ) ) )
> >
> > Is it possible that I may get a response in which the order of
> > the ACKs for the individual message sets is reversed? In other
> > words, a response that looks like this:
> >
> > ( response header ( T ( ack for S2, ack for S1 ) ) )
> >
> > I know that if a request contains multiple topics, then the topics
> > may arrive out of order in the response. So I was wondering if
> > ACKs for individual message sets within a topic can likewise
> > arrive out of order in the response.
> >
> > Thanks,
> > Dave
>
Re: question about message sets and produce requests/responses
Posted by Dave Peterson <ds...@tagged.com>.
Ok, that's all I need to know. Thanks!
On Wed, Jul 23, 2014 at 4:33 PM, Joel Koshy <jj...@gmail.com> wrote:
> Not sure if I followed the question correctly, but S1 and S2 would be
> message-sets to different partitions of T - the producer request
> object contains a map of topic-partition -> messageset). So each
> partition would get its own ack. The response object is just a map of
> partition -> response status so the order of acks in the response is
> the response status map's order.
>
> Joel
>
> On Wed, Jul 23, 2014 at 02:09:40PM -0700, Dave Peterson wrote:
> > Hello,
> >
> > I have a question about produce requests and responses.
> > Suppose I create a produce request consisting of a single topic T
> > which contains two message sets S1 and S2, with S1 preceding
> > S2. In other words, the request looks like this:
> >
> > ( request header ( T ( S1, S2 ) ) )
> >
> > Is it possible that I may get a response in which the order of
> > the ACKs for the individual message sets is reversed? In other
> > words, a response that looks like this:
> >
> > ( response header ( T ( ack for S2, ack for S1 ) ) )
> >
> > I know that if a request contains multiple topics, then the topics
> > may arrive out of order in the response. So I was wondering if
> > ACKs for individual message sets within a topic can likewise
> > arrive out of order in the response.
> >
> > Thanks,
> > Dave
>
>
Re: question about message sets and produce requests/responses
Posted by Joel Koshy <jj...@gmail.com>.
Not sure if I followed the question correctly, but S1 and S2 would be
message-sets to different partitions of T - the producer request
object contains a map of topic-partition -> messageset). So each
partition would get its own ack. The response object is just a map of
partition -> response status so the order of acks in the response is
the response status map's order.
Joel
On Wed, Jul 23, 2014 at 02:09:40PM -0700, Dave Peterson wrote:
> Hello,
>
> I have a question about produce requests and responses.
> Suppose I create a produce request consisting of a single topic T
> which contains two message sets S1 and S2, with S1 preceding
> S2. In other words, the request looks like this:
>
> ( request header ( T ( S1, S2 ) ) )
>
> Is it possible that I may get a response in which the order of
> the ACKs for the individual message sets is reversed? In other
> words, a response that looks like this:
>
> ( response header ( T ( ack for S2, ack for S1 ) ) )
>
> I know that if a request contains multiple topics, then the topics
> may arrive out of order in the response. So I was wondering if
> ACKs for individual message sets within a topic can likewise
> arrive out of order in the response.
>
> Thanks,
> Dave