You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by J A <mb...@gmail.com> on 2015/07/22 23:14:15 UTC

Round Robin Among Consumers

Hi, This is reference to stackoverflow question "
http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
"
Since Kafka 0.8 already maintains a client offset, i would like to request
a feature, where a single partition consumption can be round robin across a
set of consumers. The message delivery strategy should be an option chosen
by the consumer.

Re: Round Robin Among Consumers

Posted by J A <mb...@gmail.com>.
Ordering gurantee should be optional. If someone needs ordering, they can
always fall back on exclusive consumer strategy

On Wednesday, July 22, 2015, Ashish Singh <as...@cloudera.com> wrote:

> Hey, don't you think that would be against the basic ordering guarantees
> Kafka provides?
>
> On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com <javascript:;>>
> wrote:
>
> > Hi, This is reference to stackoverflow question "
> >
> >
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> > "
> > Since Kafka 0.8 already maintains a client offset, i would like to
> request
> > a feature, where a single partition consumption can be round robin
> across a
> > set of consumers. The message delivery strategy should be an option
> chosen
> > by the consumer.
> >
>
>
>
> --
>
> Regards,
> Ashish
>

Re: Round Robin Among Consumers

Posted by Gwen Shapira <gs...@cloudera.com>.
I'm not sure the lessons here are completely generalizable.

In this case there is a production system that works pretty well and
that has many dependencies (i.e. many producers and consumers, some
critical). I just needed to add a non-critical weekly (or possibly
just one-time, if the POC does not go well) dump to HDFS. In my
opinion, speeding up the HDFS dump didn't justify taking any risk with
the production system (and there is some risk in adding partitioning,
not large perhaps but some due-diligence regarding producers,
consumers and disk space is required). On the other hand, adding
inter-partitioning split to a MapReduce system is something I did
successfully before, so it seemed not too difficult, not too risky and
a fun challenge too.

Obviously, if there was a critical real-time stream-processing system
that required speeding up, figuring out a good way to add partitions
without screwing up anything else in the system would have been worth
it.

It all depends :)

Gwen

On Thu, Jul 23, 2015 at 9:10 PM, Ewen Cheslack-Postava
<ew...@confluent.io> wrote:
> Gwen,
>
> I'm curious about this use case. Given the Kafka -> HDFS flow, it obviously
> relates to Copycat. More generally, this could be a problem even when
> streaming data if your processing takes too long such that your consumer
> simply can't keep up with the rate at which messages are produced.
>
> The "easy" solution would have been to use more partitions since the
> problem in both the batch and streaming cases is that you need more
> processing throughput. In the case that required modifying Camus, was this
> not an option simply because making that modification was too painful
> (i.e., if there had been more partitions to start with, it might not have
> been needed at all) or because there were other constraints on partitioning?
>
> -Ewen
>
> On Thu, Jul 23, 2015 at 2:45 PM, Gwen Shapira <gs...@cloudera.com> wrote:
>
>> Agree.
>>
>> On Thu, Jul 23, 2015 at 2:43 PM, Jiangjie Qin <jq...@linkedin.com.invalid>
>> wrote:
>> > Ah, I see. Thanks for the use case, Gwen. I guess in that case it seems
>> the
>> > time to use low level consumer.
>> >
>> > Jiangjie (Becket) Qin
>> >
>> > On Thu, Jul 23, 2015 at 9:52 AM, Gwen Shapira <gs...@cloudera.com>
>> wrote:
>> >
>> >> As crazy as it sounds, there is an actual use-case there.
>> >>
>> >> Writing to HDFS can be very slow, so if you do a batch dump from a
>> >> topic to HDFS, you may want more consumers reading from the topic than
>> >> for "normal" streaming use-cases. We ended up modifying Camus to split
>> >> a partition between multiple mappers (take start and end offsets and
>> >> divide into ranges) to solve this problem. Not exactly a round-robin
>> >> but same idea.
>> >>
>> >> I think thats what J A was referring to in "decoupling consumers" -
>> >> different consumers have slightly different requirements.
>> >>
>> >> Gwen
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Jul 23, 2015 at 9:44 AM, Jiangjie Qin <jqin@linkedin.com.invalid
>> >
>> >> wrote:
>> >> > J A,
>> >> >
>> >> > It looks to me that in your case you actually want to scale the topic,
>> >> > right? Otherwise wouldn't a single consumer be enough?
>> >> >
>> >> > Jiangjie (Becket) Qin
>> >> >
>> >> > On Wed, Jul 22, 2015 at 7:39 PM, J A <mb...@gmail.com> wrote:
>> >> >
>> >> >> Why have partition at all, if I don't need to scale topic. Coupling
>> >> topic
>> >> >> scalability with consumer scalability just goes against messaging
>> >> systems
>> >> >> core principle of decoupling consumer and producers
>> >> >>
>> >> >> On Wednesday, July 22, 2015, Aditya Auradkar
>> >> >> <aa...@linkedin.com.invalid>
>> >> >> wrote:
>> >> >>
>> >> >> > Hi,
>> >> >> >
>> >> >> > Why not simply have as many partitions as the set of consumers you
>> >> want
>> >> >> to
>> >> >> > round robin across?
>> >> >> >
>> >> >> > Aditya
>> >> >> >
>> >> >> > On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
>> >> >> > <javascript:;>> wrote:
>> >> >> >
>> >> >> > > Hey, don't you think that would be against the basic ordering
>> >> >> guarantees
>> >> >> > > Kafka provides?
>> >> >> > >
>> >> >> > > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com
>> >> >> <javascript:;>>
>> >> >> > wrote:
>> >> >> > >
>> >> >> > > > Hi, This is reference to stackoverflow question "
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
>> >> >> > > > "
>> >> >> > > > Since Kafka 0.8 already maintains a client offset, i would
>> like to
>> >> >> > > request
>> >> >> > > > a feature, where a single partition consumption can be round
>> robin
>> >> >> > > across a
>> >> >> > > > set of consumers. The message delivery strategy should be an
>> >> option
>> >> >> > > chosen
>> >> >> > > > by the consumer.
>> >> >> > > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > > --
>> >> >> > >
>> >> >> > > Regards,
>> >> >> > > Ashish
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>>
>
>
>
> --
> Thanks,
> Ewen

Re: Round Robin Among Consumers

Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Gwen,

I'm curious about this use case. Given the Kafka -> HDFS flow, it obviously
relates to Copycat. More generally, this could be a problem even when
streaming data if your processing takes too long such that your consumer
simply can't keep up with the rate at which messages are produced.

The "easy" solution would have been to use more partitions since the
problem in both the batch and streaming cases is that you need more
processing throughput. In the case that required modifying Camus, was this
not an option simply because making that modification was too painful
(i.e., if there had been more partitions to start with, it might not have
been needed at all) or because there were other constraints on partitioning?

-Ewen

On Thu, Jul 23, 2015 at 2:45 PM, Gwen Shapira <gs...@cloudera.com> wrote:

> Agree.
>
> On Thu, Jul 23, 2015 at 2:43 PM, Jiangjie Qin <jq...@linkedin.com.invalid>
> wrote:
> > Ah, I see. Thanks for the use case, Gwen. I guess in that case it seems
> the
> > time to use low level consumer.
> >
> > Jiangjie (Becket) Qin
> >
> > On Thu, Jul 23, 2015 at 9:52 AM, Gwen Shapira <gs...@cloudera.com>
> wrote:
> >
> >> As crazy as it sounds, there is an actual use-case there.
> >>
> >> Writing to HDFS can be very slow, so if you do a batch dump from a
> >> topic to HDFS, you may want more consumers reading from the topic than
> >> for "normal" streaming use-cases. We ended up modifying Camus to split
> >> a partition between multiple mappers (take start and end offsets and
> >> divide into ranges) to solve this problem. Not exactly a round-robin
> >> but same idea.
> >>
> >> I think thats what J A was referring to in "decoupling consumers" -
> >> different consumers have slightly different requirements.
> >>
> >> Gwen
> >>
> >>
> >>
> >>
> >>
> >> On Thu, Jul 23, 2015 at 9:44 AM, Jiangjie Qin <jqin@linkedin.com.invalid
> >
> >> wrote:
> >> > J A,
> >> >
> >> > It looks to me that in your case you actually want to scale the topic,
> >> > right? Otherwise wouldn't a single consumer be enough?
> >> >
> >> > Jiangjie (Becket) Qin
> >> >
> >> > On Wed, Jul 22, 2015 at 7:39 PM, J A <mb...@gmail.com> wrote:
> >> >
> >> >> Why have partition at all, if I don't need to scale topic. Coupling
> >> topic
> >> >> scalability with consumer scalability just goes against messaging
> >> systems
> >> >> core principle of decoupling consumer and producers
> >> >>
> >> >> On Wednesday, July 22, 2015, Aditya Auradkar
> >> >> <aa...@linkedin.com.invalid>
> >> >> wrote:
> >> >>
> >> >> > Hi,
> >> >> >
> >> >> > Why not simply have as many partitions as the set of consumers you
> >> want
> >> >> to
> >> >> > round robin across?
> >> >> >
> >> >> > Aditya
> >> >> >
> >> >> > On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
> >> >> > <javascript:;>> wrote:
> >> >> >
> >> >> > > Hey, don't you think that would be against the basic ordering
> >> >> guarantees
> >> >> > > Kafka provides?
> >> >> > >
> >> >> > > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com
> >> >> <javascript:;>>
> >> >> > wrote:
> >> >> > >
> >> >> > > > Hi, This is reference to stackoverflow question "
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> >> >> > > > "
> >> >> > > > Since Kafka 0.8 already maintains a client offset, i would
> like to
> >> >> > > request
> >> >> > > > a feature, where a single partition consumption can be round
> robin
> >> >> > > across a
> >> >> > > > set of consumers. The message delivery strategy should be an
> >> option
> >> >> > > chosen
> >> >> > > > by the consumer.
> >> >> > > >
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > --
> >> >> > >
> >> >> > > Regards,
> >> >> > > Ashish
> >> >> > >
> >> >> >
> >> >>
> >>
>



-- 
Thanks,
Ewen

Re: Round Robin Among Consumers

Posted by Gwen Shapira <gs...@cloudera.com>.
Agree.

On Thu, Jul 23, 2015 at 2:43 PM, Jiangjie Qin <jq...@linkedin.com.invalid> wrote:
> Ah, I see. Thanks for the use case, Gwen. I guess in that case it seems the
> time to use low level consumer.
>
> Jiangjie (Becket) Qin
>
> On Thu, Jul 23, 2015 at 9:52 AM, Gwen Shapira <gs...@cloudera.com> wrote:
>
>> As crazy as it sounds, there is an actual use-case there.
>>
>> Writing to HDFS can be very slow, so if you do a batch dump from a
>> topic to HDFS, you may want more consumers reading from the topic than
>> for "normal" streaming use-cases. We ended up modifying Camus to split
>> a partition between multiple mappers (take start and end offsets and
>> divide into ranges) to solve this problem. Not exactly a round-robin
>> but same idea.
>>
>> I think thats what J A was referring to in "decoupling consumers" -
>> different consumers have slightly different requirements.
>>
>> Gwen
>>
>>
>>
>>
>>
>> On Thu, Jul 23, 2015 at 9:44 AM, Jiangjie Qin <jq...@linkedin.com.invalid>
>> wrote:
>> > J A,
>> >
>> > It looks to me that in your case you actually want to scale the topic,
>> > right? Otherwise wouldn't a single consumer be enough?
>> >
>> > Jiangjie (Becket) Qin
>> >
>> > On Wed, Jul 22, 2015 at 7:39 PM, J A <mb...@gmail.com> wrote:
>> >
>> >> Why have partition at all, if I don't need to scale topic. Coupling
>> topic
>> >> scalability with consumer scalability just goes against messaging
>> systems
>> >> core principle of decoupling consumer and producers
>> >>
>> >> On Wednesday, July 22, 2015, Aditya Auradkar
>> >> <aa...@linkedin.com.invalid>
>> >> wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > Why not simply have as many partitions as the set of consumers you
>> want
>> >> to
>> >> > round robin across?
>> >> >
>> >> > Aditya
>> >> >
>> >> > On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
>> >> > <javascript:;>> wrote:
>> >> >
>> >> > > Hey, don't you think that would be against the basic ordering
>> >> guarantees
>> >> > > Kafka provides?
>> >> > >
>> >> > > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com
>> >> <javascript:;>>
>> >> > wrote:
>> >> > >
>> >> > > > Hi, This is reference to stackoverflow question "
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
>> >> > > > "
>> >> > > > Since Kafka 0.8 already maintains a client offset, i would like to
>> >> > > request
>> >> > > > a feature, where a single partition consumption can be round robin
>> >> > > across a
>> >> > > > set of consumers. The message delivery strategy should be an
>> option
>> >> > > chosen
>> >> > > > by the consumer.
>> >> > > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > >
>> >> > > Regards,
>> >> > > Ashish
>> >> > >
>> >> >
>> >>
>>

Re: Round Robin Among Consumers

Posted by Jiangjie Qin <jq...@linkedin.com.INVALID>.
Ah, I see. Thanks for the use case, Gwen. I guess in that case it seems the
time to use low level consumer.

Jiangjie (Becket) Qin

On Thu, Jul 23, 2015 at 9:52 AM, Gwen Shapira <gs...@cloudera.com> wrote:

> As crazy as it sounds, there is an actual use-case there.
>
> Writing to HDFS can be very slow, so if you do a batch dump from a
> topic to HDFS, you may want more consumers reading from the topic than
> for "normal" streaming use-cases. We ended up modifying Camus to split
> a partition between multiple mappers (take start and end offsets and
> divide into ranges) to solve this problem. Not exactly a round-robin
> but same idea.
>
> I think thats what J A was referring to in "decoupling consumers" -
> different consumers have slightly different requirements.
>
> Gwen
>
>
>
>
>
> On Thu, Jul 23, 2015 at 9:44 AM, Jiangjie Qin <jq...@linkedin.com.invalid>
> wrote:
> > J A,
> >
> > It looks to me that in your case you actually want to scale the topic,
> > right? Otherwise wouldn't a single consumer be enough?
> >
> > Jiangjie (Becket) Qin
> >
> > On Wed, Jul 22, 2015 at 7:39 PM, J A <mb...@gmail.com> wrote:
> >
> >> Why have partition at all, if I don't need to scale topic. Coupling
> topic
> >> scalability with consumer scalability just goes against messaging
> systems
> >> core principle of decoupling consumer and producers
> >>
> >> On Wednesday, July 22, 2015, Aditya Auradkar
> >> <aa...@linkedin.com.invalid>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > Why not simply have as many partitions as the set of consumers you
> want
> >> to
> >> > round robin across?
> >> >
> >> > Aditya
> >> >
> >> > On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
> >> > <javascript:;>> wrote:
> >> >
> >> > > Hey, don't you think that would be against the basic ordering
> >> guarantees
> >> > > Kafka provides?
> >> > >
> >> > > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com
> >> <javascript:;>>
> >> > wrote:
> >> > >
> >> > > > Hi, This is reference to stackoverflow question "
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> >> > > > "
> >> > > > Since Kafka 0.8 already maintains a client offset, i would like to
> >> > > request
> >> > > > a feature, where a single partition consumption can be round robin
> >> > > across a
> >> > > > set of consumers. The message delivery strategy should be an
> option
> >> > > chosen
> >> > > > by the consumer.
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > >
> >> > > Regards,
> >> > > Ashish
> >> > >
> >> >
> >>
>

Re: Round Robin Among Consumers

Posted by Gwen Shapira <gs...@cloudera.com>.
As crazy as it sounds, there is an actual use-case there.

Writing to HDFS can be very slow, so if you do a batch dump from a
topic to HDFS, you may want more consumers reading from the topic than
for "normal" streaming use-cases. We ended up modifying Camus to split
a partition between multiple mappers (take start and end offsets and
divide into ranges) to solve this problem. Not exactly a round-robin
but same idea.

I think thats what J A was referring to in "decoupling consumers" -
different consumers have slightly different requirements.

Gwen





On Thu, Jul 23, 2015 at 9:44 AM, Jiangjie Qin <jq...@linkedin.com.invalid> wrote:
> J A,
>
> It looks to me that in your case you actually want to scale the topic,
> right? Otherwise wouldn't a single consumer be enough?
>
> Jiangjie (Becket) Qin
>
> On Wed, Jul 22, 2015 at 7:39 PM, J A <mb...@gmail.com> wrote:
>
>> Why have partition at all, if I don't need to scale topic. Coupling topic
>> scalability with consumer scalability just goes against messaging systems
>> core principle of decoupling consumer and producers
>>
>> On Wednesday, July 22, 2015, Aditya Auradkar
>> <aa...@linkedin.com.invalid>
>> wrote:
>>
>> > Hi,
>> >
>> > Why not simply have as many partitions as the set of consumers you want
>> to
>> > round robin across?
>> >
>> > Aditya
>> >
>> > On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
>> > <javascript:;>> wrote:
>> >
>> > > Hey, don't you think that would be against the basic ordering
>> guarantees
>> > > Kafka provides?
>> > >
>> > > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com
>> <javascript:;>>
>> > wrote:
>> > >
>> > > > Hi, This is reference to stackoverflow question "
>> > > >
>> > > >
>> > >
>> >
>> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
>> > > > "
>> > > > Since Kafka 0.8 already maintains a client offset, i would like to
>> > > request
>> > > > a feature, where a single partition consumption can be round robin
>> > > across a
>> > > > set of consumers. The message delivery strategy should be an option
>> > > chosen
>> > > > by the consumer.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > >
>> > > Regards,
>> > > Ashish
>> > >
>> >
>>

Re: Round Robin Among Consumers

Posted by Jiangjie Qin <jq...@linkedin.com.INVALID>.
J A,

It looks to me that in your case you actually want to scale the topic,
right? Otherwise wouldn't a single consumer be enough?

Jiangjie (Becket) Qin

On Wed, Jul 22, 2015 at 7:39 PM, J A <mb...@gmail.com> wrote:

> Why have partition at all, if I don't need to scale topic. Coupling topic
> scalability with consumer scalability just goes against messaging systems
> core principle of decoupling consumer and producers
>
> On Wednesday, July 22, 2015, Aditya Auradkar
> <aa...@linkedin.com.invalid>
> wrote:
>
> > Hi,
> >
> > Why not simply have as many partitions as the set of consumers you want
> to
> > round robin across?
> >
> > Aditya
> >
> > On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
> > <javascript:;>> wrote:
> >
> > > Hey, don't you think that would be against the basic ordering
> guarantees
> > > Kafka provides?
> > >
> > > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com
> <javascript:;>>
> > wrote:
> > >
> > > > Hi, This is reference to stackoverflow question "
> > > >
> > > >
> > >
> >
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> > > > "
> > > > Since Kafka 0.8 already maintains a client offset, i would like to
> > > request
> > > > a feature, where a single partition consumption can be round robin
> > > across a
> > > > set of consumers. The message delivery strategy should be an option
> > > chosen
> > > > by the consumer.
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Regards,
> > > Ashish
> > >
> >
>

Re: Round Robin Among Consumers

Posted by J A <mb...@gmail.com>.
Why have partition at all, if I don't need to scale topic. Coupling topic
scalability with consumer scalability just goes against messaging systems
core principle of decoupling consumer and producers

On Wednesday, July 22, 2015, Aditya Auradkar <aa...@linkedin.com.invalid>
wrote:

> Hi,
>
> Why not simply have as many partitions as the set of consumers you want to
> round robin across?
>
> Aditya
>
> On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <asingh@cloudera.com
> <javascript:;>> wrote:
>
> > Hey, don't you think that would be against the basic ordering guarantees
> > Kafka provides?
> >
> > On Wed, Jul 22, 2015 at 2:14 PM, J A <mbatth.is@gmail.com <javascript:;>>
> wrote:
> >
> > > Hi, This is reference to stackoverflow question "
> > >
> > >
> >
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> > > "
> > > Since Kafka 0.8 already maintains a client offset, i would like to
> > request
> > > a feature, where a single partition consumption can be round robin
> > across a
> > > set of consumers. The message delivery strategy should be an option
> > chosen
> > > by the consumer.
> > >
> >
> >
> >
> > --
> >
> > Regards,
> > Ashish
> >
>

Re: Round Robin Among Consumers

Posted by Aditya Auradkar <aa...@linkedin.com.INVALID>.
Hi,

Why not simply have as many partitions as the set of consumers you want to
round robin across?

Aditya

On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh <as...@cloudera.com> wrote:

> Hey, don't you think that would be against the basic ordering guarantees
> Kafka provides?
>
> On Wed, Jul 22, 2015 at 2:14 PM, J A <mb...@gmail.com> wrote:
>
> > Hi, This is reference to stackoverflow question "
> >
> >
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> > "
> > Since Kafka 0.8 already maintains a client offset, i would like to
> request
> > a feature, where a single partition consumption can be round robin
> across a
> > set of consumers. The message delivery strategy should be an option
> chosen
> > by the consumer.
> >
>
>
>
> --
>
> Regards,
> Ashish
>

Re: Round Robin Among Consumers

Posted by Ashish Singh <as...@cloudera.com>.
Hey, don't you think that would be against the basic ordering guarantees
Kafka provides?

On Wed, Jul 22, 2015 at 2:14 PM, J A <mb...@gmail.com> wrote:

> Hi, This is reference to stackoverflow question "
>
> http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers
> "
> Since Kafka 0.8 already maintains a client offset, i would like to request
> a feature, where a single partition consumption can be round robin across a
> set of consumers. The message delivery strategy should be an option chosen
> by the consumer.
>



-- 

Regards,
Ashish