Doubt about ShuffleGrouping and LocalOrShuffleGrouping

[Mohit Goyal <it...@gmail.com>]

Hi,

Can someone here tell me the difference between shuffleGrouping and
localOrShuffleGrouping?
My use case is: I have supervisor running in 3 different instances and
workers/tasks distributed among them.
My understanding for grouping is: if I use shufflegrouping it will send
tuple to any task of bolt be in same machine or different. But if I use
localOrShuffleGrouping, it will try to send tuple to such task of next bolt
which lies in same machine, and if its overloaded, then it'll send it to
task residing in other machine.
Please let me know the difference. Thanks in advance.

Regards,
Mohit Goyal

Re: Doubt about ShuffleGrouping and LocalOrShuffleGrouping

[Ethan Li <et...@gmail.com>]

Hi Mohit,

Your understanding about ShuffleGrouping is correct. 

For LocalOrShuffleGrouping,  the upstream spout/bolt will only send tuples to the downstream bolt on the same worker (not machine) if it can. Otherwise, it will be just  like ShuffleGrouping. 

Code: https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/GrouperFactory.java#L86-L88 <https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/GrouperFactory.java#L86-L88>


Thanks
Ethan


> On May 14, 2019, at 11:40 AM, Mohit Goyal <it...@gmail.com> wrote:
> 
> Hi,
> 
> Can someone here tell me the difference between shuffleGrouping and localOrShuffleGrouping?
> My use case is: I have supervisor running in 3 different instances and workers/tasks distributed among them.
> My understanding for grouping is: if I use shufflegrouping it will send tuple to any task of bolt be in same machine or different. But if I use localOrShuffleGrouping, it will try to send tuple to such task of next bolt which lies in same machine, and if its overloaded, then it'll send it to task residing in other machine.
> Please let me know the difference. Thanks in advance.
> 
> Regards,
> Mohit Goyal