Message distribution among workers

[anshu shukla <an...@gmail.com>]

Hello ,


I am unable to find in the documentation that  how messages are distributed
to the workers of same bolt . *Are they distributed equally among all the
workers ?*


*This will impact the performance if thread  of same bolt are not
uniformly  distributed among multiple workers.*

-- 
Thanks & Regards,
Anshu Shukla

Re: Message distribution among workers

[Kevin Peek <kp...@salesforce.com>]

So, how the messages are distributed depends on the type of Stream Grouping
you select. Probably the most common choice is ShuffleGrouping which does
send an equal number of tuples to each bolt instance.

The various types of stream groupings are described here (scroll down to
the Stream Groupings heading):
http://storm.apache.org/releases/current/Concepts.html

Storm 1.x added the LoadAwareShuffleGrouping, which distributes tuples
randomly, but gives instances under light load a higher probability of
being chosen. It is missing from the document above, but perhaps it would
be helpful to you if you are worried about how many tuples each Worker
gets.

LoadAwareShuffleGrouping:
https://github.com/apache/storm/blob/cd5c9e8f904205a6ca6eee9222ca954ca8b37ec3/storm-core/src/jvm/org/apache/storm/grouping/LoadAwareShuffleGrouping.java

On Tue, Dec 13, 2016 at 12:20 AM, anshu shukla <an...@gmail.com>
wrote:

> Hello ,
>
>
> I am unable to find in the documentation that  how messages are
> distributed to the workers of same bolt . *Are they distributed equally
> among all the workers ?*
>
>
> *This will impact the performance if thread  of same bolt are not
> uniformly  distributed among multiple workers.*
>
> --
> Thanks & Regards,
> Anshu Shukla
>