Apache Storm ( Stream Processing ) vs Traditional Queues/Workers

[Mohit Gupta <su...@gmail.com>]

Hi,

Sorry for sounding naive, but I am trying to understand the use cases of
storm. The rationale given in storm documentation is mainly the pain in
deploying/managing queues & workers and partitioning the work between the
workers. (http://storm.apache.org/releases/current/Rationale.html )

But now that we have reliable and established queues like Kafka & others,
it looks simpler to me to have multiple queues and multiple workers
depending upon load processing messages from each queue.

Eg. Word Count Push tweets to a queue ( q1 ) -> multiple workers processing
this queue and pushing words to another queue ( q2 ) -> a worker generating
word count.

Can you please share an example of a production topology (with slight
comparison) where it would be complex to handle to traditional queue/worker
model but easier with storm?

Thanks & Regards
Mohit

Re: Apache Storm ( Stream Processing ) vs Traditional Queues/Workers

[Anshul Gupta <an...@gmail.com>]

Atleast and atmost once guarantees is the key, its hard to achieve that in
your setup.

On Tue, Apr 18, 2017 at 1:30 AM Mohit Gupta <su...@gmail.com>
wrote:

> Hi,
>
> Sorry for sounding naive, but I am trying to understand the use cases of
> storm. The rationale given in storm documentation is mainly the pain in
> deploying/managing queues & workers and partitioning the work between the
> workers. (http://storm.apache.org/releases/current/Rationale.html )
>
> But now that we have reliable and established queues like Kafka & others,
> it looks simpler to me to have multiple queues and multiple workers
> depending upon load processing messages from each queue.
>
> Eg. Word Count Push tweets to a queue ( q1 ) -> multiple workers
> processing this queue and pushing words to another queue ( q2 ) -> a worker
> generating word count.
>
> Can you please share an example of a production topology (with slight
> comparison) where it would be complex to handle to traditional queue/worker
> model but easier with storm?
>
> Thanks & Regards
> Mohit
>