Store state data in zookeeper from a spout or bolt

["harrigan.matthew" <ha...@gmail.com>]

Hello,

I need to store a small amount of state data from a spout or bolt.  I think zookeeper would perform this well.  Is it feasible to interact with zookeeper from inside a spout or bolt, how should it be implemented, and what are the gotchas to avoid?  I am writing in Java and would prefer to use the curator api.  Note that stack overflow has a related unanswered question at http://stackoverflow.com/questions/28549722/access-to-zookeeper-from-within-storm-bolt.

Thank you

Re: Store state data in zookeeper from a spout or bolt

[Javier Gonzalez <ja...@gmail.com>]

It can be done, with the curator api. We did it in the middle of a PoC a
month ago or so, to store some history that would be needed to detect if an
incoming event was already processed. It performed well. Unfortunately, I
can't share any code as that goes against my contract. I think what we did
was to look inside the actual Storm code to see how they were interfacing
with zookeeper, and mimicing that :)

Regards,
JG

On Fri, Apr 10, 2015 at 5:29 AM, harrigan.matthew <
harrigan.matthew@gmail.com> wrote:

> Hello,
>
> I need to store a small amount of state data from a spout or bolt.  I
> think zookeeper would perform this well.  Is it feasible to interact with
> zookeeper from inside a spout or bolt, how should it be implemented, and
> what are the gotchas to avoid?  I am writing in Java and would prefer to
> use the curator api.  Note that stack overflow has a related unanswered
> question at
> http://stackoverflow.com/questions/28549722/access-to-zookeeper-from-within-storm-bolt
> .
>
> Thank you
>



-- 
Javier González Nicolini