You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Pandian R <pa...@gmail.com> on 2015/01/28 15:55:02 UTC
startBatch and finishBatch for Trident functions
Hi,
I am implementing a trident topology and using functions at many steps(as
alternative to traditional bolts). But, I don't find a startBatch and
finishBatch function. I need to do some operations per batch within a
function. Function has prepare and cleanup which is called per tuple. Can
someone help how to implement startBatch and finishBatch functions in a
Function ?
Thanks !
--
Regards,
Pandian
Re: startBatch and finishBatch for Trident functions
Posted by Pandian R <pa...@gmail.com>.
Thanks. Implemented aggregator to emit the tuple as such and used the same
fields of the old stream as the output fields also.
On Thu, Jan 29, 2015 at 6:00 PM, Brunner, Bill <bi...@baml.com>
wrote:
> An aggregator just collects all the tuples in a batch. You don’t even
> have to do anything with them inside the aggregator. When the batch has
> been fully aggregated, then it emits whatever you tell it to from the
> complete() method. Sure, the output field name is different, but it’s
> still the same tuple.
>
>
>
> *From:* Pandian R [mailto:pandian4mail@gmail.com]
> *Sent:* Thursday, January 29, 2015 4:21 AM
> *To:* user; nathan@nathanmarz.com
> *Subject:* Re: startBatch and finishBatch for Trident functions
>
>
>
> Thanks. Tried using an Aggregator first, but using aggregator loses the
> original fields in the tuple and forwards only the new fields to the output
> stream.
>
>
>
> On Wed, Jan 28, 2015 at 8:44 PM, Nathan Marz <na...@nathanmarz.com>
> wrote:
>
> You should use an Aggregator if you want to process many tuples within a
> batch, or a State and a StateUpdater if you want to process many tuples
> across batches.
>
>
>
> On Wed, Jan 28, 2015 at 9:55 AM, Pandian R <pa...@gmail.com> wrote:
>
> Hi,
>
> I am implementing a trident topology and using functions at many steps(as
> alternative to traditional bolts). But, I don't find a startBatch and
> finishBatch function. I need to do some operations per batch within a
> function. Function has prepare and cleanup which is called per tuple. Can
> someone help how to implement startBatch and finishBatch functions in a
> Function ?
>
> Thanks !
>
>
> --
>
> Regards,
>
> Pandian
>
>
>
>
>
> --
>
> Twitter: @nathanmarz
>
> http://nathanmarz.com
>
>
>
>
> --
>
> Regards,
>
> Pandian
> ------------------------------
> This message, and any attachments, is for the intended recipient(s) only,
> may contain information that is privileged, confidential and/or proprietary
> and subject to important terms and conditions available at
> http://www.bankofamerica.com/emaildisclaimer. If you are not the intended
> recipient, please delete this message.
>
--
Regards,
Pandian
RE: startBatch and finishBatch for Trident functions
Posted by "Brunner, Bill" <bi...@baml.com>.
An aggregator just collects all the tuples in a batch. You don’t even have to do anything with them inside the aggregator. When the batch has been fully aggregated, then it emits whatever you tell it to from the complete() method. Sure, the output field name is different, but it’s still the same tuple.
From: Pandian R [mailto:pandian4mail@gmail.com]
Sent: Thursday, January 29, 2015 4:21 AM
To: user; nathan@nathanmarz.com
Subject: Re: startBatch and finishBatch for Trident functions
Thanks. Tried using an Aggregator first, but using aggregator loses the original fields in the tuple and forwards only the new fields to the output stream.
On Wed, Jan 28, 2015 at 8:44 PM, Nathan Marz <na...@nathanmarz.com>> wrote:
You should use an Aggregator if you want to process many tuples within a batch, or a State and a StateUpdater if you want to process many tuples across batches.
On Wed, Jan 28, 2015 at 9:55 AM, Pandian R <pa...@gmail.com>> wrote:
Hi,
I am implementing a trident topology and using functions at many steps(as alternative to traditional bolts). But, I don't find a startBatch and finishBatch function. I need to do some operations per batch within a function. Function has prepare and cleanup which is called per tuple. Can someone help how to implement startBatch and finishBatch functions in a Function ?
Thanks !
--
Regards,
Pandian
--
Twitter: @nathanmarz
http://nathanmarz.com<http://nathanmarz.com/>
--
Regards,
Pandian
----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
Re: startBatch and finishBatch for Trident functions
Posted by Pandian R <pa...@gmail.com>.
Thanks. Tried using an Aggregator first, but using aggregator loses the
original fields in the tuple and forwards only the new fields to the output
stream.
On Wed, Jan 28, 2015 at 8:44 PM, Nathan Marz <na...@nathanmarz.com> wrote:
> You should use an Aggregator if you want to process many tuples within a
> batch, or a State and a StateUpdater if you want to process many tuples
> across batches.
>
> On Wed, Jan 28, 2015 at 9:55 AM, Pandian R <pa...@gmail.com> wrote:
>
>> Hi,
>>
>> I am implementing a trident topology and using functions at many steps(as
>> alternative to traditional bolts). But, I don't find a startBatch and
>> finishBatch function. I need to do some operations per batch within a
>> function. Function has prepare and cleanup which is called per tuple. Can
>> someone help how to implement startBatch and finishBatch functions in a
>> Function ?
>>
>> Thanks !
>>
>> --
>> Regards,
>> Pandian
>>
>
>
>
> --
> Twitter: @nathanmarz
> http://nathanmarz.com
>
--
Regards,
Pandian
Re: startBatch and finishBatch for Trident functions
Posted by Nathan Marz <na...@nathanmarz.com>.
You should use an Aggregator if you want to process many tuples within a
batch, or a State and a StateUpdater if you want to process many tuples
across batches.
On Wed, Jan 28, 2015 at 9:55 AM, Pandian R <pa...@gmail.com> wrote:
> Hi,
>
> I am implementing a trident topology and using functions at many steps(as
> alternative to traditional bolts). But, I don't find a startBatch and
> finishBatch function. I need to do some operations per batch within a
> function. Function has prepare and cleanup which is called per tuple. Can
> someone help how to implement startBatch and finishBatch functions in a
> Function ?
>
> Thanks !
>
> --
> Regards,
> Pandian
>
--
Twitter: @nathanmarz
http://nathanmarz.com