You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Johannes Hugo Kitschke <jo...@rwth-aachen.de> on 2015/04/27 17:14:30 UTC

Clarification concerning parallelism...

Hi,

I have a few questions:

1. 'Example of a running topology' in the Wiki: 
http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html

The yellow bolt is configured with a parallelism hint of 6. Since no 
number of tasks is specified, it imlicitly is equal to parallelism hint. 
Now in the next section the number of executors is increased to 10 which 
does not make too much sense since # of tasks = const = 6, right?

2. The topology is rebalanced to use 5 instead of 2 workers. This 
assumes, that 3 additional worker slots are available on the cluster!?

3. I read the Storm @ Twitter paper where they write 'A task is strictly 
bound to an executor because that assignment is currently static.' This 
is outdated, right? How else would the rebalancing work?

4. An executor can only execute tasks of the same component. These tasks 
are executed in serial. Right?

Thanks,
  Johannes

Re: Clarification concerning parallelism...

Posted by "Matthias J. Sax" <mj...@informatik.hu-berlin.de>.
From my understanding, I agree with all of what you are saying.


-> Point 3: the paper does not tell which Storm version is used; the
concept of executors was introduce with version 0.8.0

-> Point 4: concurrently would be the correct term, though ;)


-Matthias


On 04/27/2015 05:14 PM, Johannes Hugo Kitschke wrote:
> Hi,
> 
> I have a few questions:
> 
> 1. 'Example of a running topology' in the Wiki:
> http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html
> 
> 
> The yellow bolt is configured with a parallelism hint of 6. Since no
> number of tasks is specified, it imlicitly is equal to parallelism hint.
> Now in the next section the number of executors is increased to 10 which
> does not make too much sense since # of tasks = const = 6, right?
> 
> 2. The topology is rebalanced to use 5 instead of 2 workers. This
> assumes, that 3 additional worker slots are available on the cluster!?
> 
> 3. I read the Storm @ Twitter paper where they write 'A task is strictly
> bound to an executor because that assignment is currently static.' This
> is outdated, right? How else would the rebalancing work?
> 
> 4. An executor can only execute tasks of the same component. These tasks
> are executed in serial. Right?
> 
> Thanks,
>  Johannes