Kafka 0.8.2 ConsumerGroup Example

[Joe San <co...@gmail.com>]

I'm following the ConsumerGroup example,

https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example

How can I specify the batch size of the messages that I want to consume? I
see that if I use the SimpleConsumer, I can specify a size that I want to
read. How can I do it here with the ConsumerGroup? I want to be able to
consume let's say 1000 records, process them and so on...

Re: Kafka 0.8.2 ConsumerGroup Example

[Sharninder <sh...@gmail.com>]

How you do this in the high level consumer is slightly different. First,
set the auto offset commit property to false. Then you can keep reading
messages from kafka and when you're done filling up your buffer, commit
offsets manually. The following link gives an example:

http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/

--
Sharninder


On Wed, Feb 10, 2016 at 9:11 PM, Joe San <co...@gmail.com> wrote:

> I'm following the ConsumerGroup example,
>
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
>
> How can I specify the batch size of the messages that I want to consume? I
> see that if I use the SimpleConsumer, I can specify a size that I want to
> read. How can I do it here with the ConsumerGroup? I want to be able to
> consume let's say 1000 records, process them and so on...
>



-- 
--
Sharninder