You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by Gaurav <ga...@gmail.com> on 2018/12/31 03:44:48 UTC

How to configure Kafka spout to fetch n tuples (reading in batches).

Hi All,
I am using 1.2.2 Kafka-spout, I am not able to find the configuration which
will allow me to read Kafka in batches. Any leads will be appreciated.

-- 
*Gaurav*
Mob: 9845074451

Re: How to configure Kafka spout to fetch n tuples (reading in batches).

Posted by Hugo Louro <hm...@gmail.com>.

Hi, what do you mean by read in batches ? The KafkaSpout uses the KafkaConsumer#poll() method underneath, which polls batches of records from Kafka. However the contract of the KafkaSpout sends one record downstream per call to nextTuple().

To send batches of records downstream you may want to take a look at the KafkaTridentSpout.

Hugo

> On Dec 30, 2018, at 7:44 PM, Gaurav <ga...@gmail.com> wrote:
> 
> Hi All,
> I am using 1.2.2 Kafka-spout, I am not able to find the configuration which
> will allow me to read Kafka in batches. Any leads will be appreciated.
> 
> -- 
> *Gaurav*
> Mob: 9845074451