Reusing Bolts in Trident

[Tarek Amr <ta...@meedan.net>]

Hello,

In one of the project I am working on, we use a basic topology that include
a spot that collects some real-time updates, then we do some processing on
that stream in a number of the intermediate bolts before saving them to the
database.

Now, we want to generate some statistics for this real time data. Thus, I
thought of creating another Trident topology, that uses the same spout in
order to get such statistics. The problem here, is that I need my
statistics to also include details that are based on the processing done in
the bolts, and as you all know, you cannot have bolts in a Trident
topology, in addition to that, my bolts are written in Python.

Basically, I'd like to ask for your best practice in such case. I want to
find a way to reuse my code, rather than porting all the python code for
the bolts to Java to be used inside a trident function/filter. Is there a
way for example to call the storm bolts from within the trident functions,
using some kind of python adapters? Are there more reasonable ways to
achieve what I wanna do here, that I don't know?




*Bonus question that you may ignore for now:* Is there a quick way to
convert a non-transactional spout into a transactional one without the need
to re-write the spoit?




---

Cheers,
Tarek Amr

Email: tamr@meedan.net