You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Oleg Zhurakousky <oz...@hortonworks.com> on 2016/04/12 14:56:03 UTC
Control the amount of messages batched up by KafkaConsumer.poll()
Is there a way to specify in KafkaConsumer up to how many messages do I want o receive? I am operation under premise that Consumer.poll(..) returns a batch of messages, but not sure if there is a way to control batch amount.
Cheers
Oleg
Re: Control the amount of messages batched up by KafkaConsumer.poll()
Posted by Manikumar Reddy <ma...@gmail.com>.
Yes.
Manikumar
On Tue, Apr 12, 2016 at 6:59 PM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com> wrote:
> Thanks Manikumar
>
> So, but for now I guess the only way to at least influence it (as I’ve
> observed) would be ‘max.partition.fetch.bytes’, correct?
>
> Cheers
> Oleg
> > On Apr 12, 2016, at 9:22 AM, Manikumar Reddy <ma...@gmail.com>
> wrote:
> >
> > New consumer config property "max.poll.records" is getting introduced
> in
> > upcoming 0.10 release.
> > This property can be used to control the no. of records in each poll.
> >
> >
> > Manikumar
> >
> > On Tue, Apr 12, 2016 at 6:26 PM, Oleg Zhurakousky <
> > ozhurakousky@hortonworks.com> wrote:
> >
> >> Is there a way to specify in KafkaConsumer up to how many messages do I
> >> want o receive? I am operation under premise that Consumer.poll(..)
> returns
> >> a batch of messages, but not sure if there is a way to control batch
> amount.
> >>
> >> Cheers
> >> Oleg
> >>
>
>
Re: Control the amount of messages batched up by
KafkaConsumer.poll()
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Thanks Manikumar
So, but for now I guess the only way to at least influence it (as I’ve observed) would be ‘max.partition.fetch.bytes’, correct?
Cheers
Oleg
> On Apr 12, 2016, at 9:22 AM, Manikumar Reddy <ma...@gmail.com> wrote:
>
> New consumer config property "max.poll.records" is getting introduced in
> upcoming 0.10 release.
> This property can be used to control the no. of records in each poll.
>
>
> Manikumar
>
> On Tue, Apr 12, 2016 at 6:26 PM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com> wrote:
>
>> Is there a way to specify in KafkaConsumer up to how many messages do I
>> want o receive? I am operation under premise that Consumer.poll(..) returns
>> a batch of messages, but not sure if there is a way to control batch amount.
>>
>> Cheers
>> Oleg
>>
Re: Control the amount of messages batched up by KafkaConsumer.poll()
Posted by Manikumar Reddy <ma...@gmail.com>.
New consumer config property "max.poll.records" is getting introduced in
upcoming 0.10 release.
This property can be used to control the no. of records in each poll.
Manikumar
On Tue, Apr 12, 2016 at 6:26 PM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com> wrote:
> Is there a way to specify in KafkaConsumer up to how many messages do I
> want o receive? I am operation under premise that Consumer.poll(..) returns
> a batch of messages, but not sure if there is a way to control batch amount.
>
> Cheers
> Oleg
>