You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Raphael Hsieh <ra...@gmail.com> on 2014/05/27 19:49:27 UTC
PersistentAggregate
>From my understanding, PersistentAggregate should first aggregate the
batch, then once the batch has finished aggregating, send it to whatever
datastore is specified.
Is this the case ? Or will the Persistent Aggregate use the external
datastore in order to do the aggregations ?
--
Raphael Hsieh
Re: PersistentAggregate
Posted by "Cody A. Ray" <co...@gmail.com>.
You may find this helpful to understanding persistantAggregate works
internally.
http://svendvanderveken.wordpress.com/2013/07/30/scalable-real-time-state-update-with-storm/
-Cody
On Tue, May 27, 2014 at 1:53 PM, Danijel Schiavuzzi
<da...@schiavuzzi.com>wrote:
> The aggregations are done by Storm. persistentAggregate provides only a
> means to access the datastore to get the current aggregate for the
> specified key (using the IBackingMap's multiGet() implementation),
> provide that aggregate as the input to the Aggregator implementation along
> with other same-keyed tuples in the current batch (the
> groupBy/GroupedStream), and persist it back to the database using
> multiPut().
>
>
> On Tuesday, May 27, 2014, Raphael Hsieh <ra...@gmail.com> wrote:
>
>> From my understanding, PersistentAggregate should first aggregate the
>> batch, then once the batch has finished aggregating, send it to whatever
>> datastore is specified.
>>
>> Is this the case ? Or will the Persistent Aggregate use the external
>> datastore in order to do the aggregations ?
>>
>> --
>> Raphael Hsieh
>>
>>
>>
>
>
> --
> Danijel Schiavuzzi
>
> E: danijel@schiavuzzi.com
> W: www.schiavuzzi.com
> T: +385989035562
> Skype: danijels7
>
--
Cody A. Ray, LEED AP
cody.a.ray@gmail.com
215.501.7891
Re: PersistentAggregate
Posted by Danijel Schiavuzzi <da...@schiavuzzi.com>.
The aggregations are done by Storm. persistentAggregate provides only a
means to access the datastore to get the current aggregate for the
specified key (using the IBackingMap's multiGet() implementation),
provide that aggregate as the input to the Aggregator implementation along
with other same-keyed tuples in the current batch (the
groupBy/GroupedStream), and persist it back to the database using
multiPut().
On Tuesday, May 27, 2014, Raphael Hsieh <ra...@gmail.com> wrote:
> From my understanding, PersistentAggregate should first aggregate the
> batch, then once the batch has finished aggregating, send it to whatever
> datastore is specified.
>
> Is this the case ? Or will the Persistent Aggregate use the external
> datastore in order to do the aggregations ?
>
> --
> Raphael Hsieh
>
>
>
--
Danijel Schiavuzzi
E: danijel@schiavuzzi.com
W: www.schiavuzzi.com
T: +385989035562
Skype: danijels7