You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Johannes Hugo Kitschke <jo...@rwth-aachen.de> on 2015/04/27 17:14:30 UTC
Clarification concerning parallelism...
Hi,
I have a few questions:
1. 'Example of a running topology' in the Wiki:
http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html
The yellow bolt is configured with a parallelism hint of 6. Since no
number of tasks is specified, it imlicitly is equal to parallelism hint.
Now in the next section the number of executors is increased to 10 which
does not make too much sense since # of tasks = const = 6, right?
2. The topology is rebalanced to use 5 instead of 2 workers. This
assumes, that 3 additional worker slots are available on the cluster!?
3. I read the Storm @ Twitter paper where they write 'A task is strictly
bound to an executor because that assignment is currently static.' This
is outdated, right? How else would the rebalancing work?
4. An executor can only execute tasks of the same component. These tasks
are executed in serial. Right?
Thanks,
Johannes
Re: Clarification concerning parallelism...
Posted by "Matthias J. Sax" <mj...@informatik.hu-berlin.de>.
From my understanding, I agree with all of what you are saying.
-> Point 3: the paper does not tell which Storm version is used; the
concept of executors was introduce with version 0.8.0
-> Point 4: concurrently would be the correct term, though ;)
-Matthias
On 04/27/2015 05:14 PM, Johannes Hugo Kitschke wrote:
> Hi,
>
> I have a few questions:
>
> 1. 'Example of a running topology' in the Wiki:
> http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html
>
>
> The yellow bolt is configured with a parallelism hint of 6. Since no
> number of tasks is specified, it imlicitly is equal to parallelism hint.
> Now in the next section the number of executors is increased to 10 which
> does not make too much sense since # of tasks = const = 6, right?
>
> 2. The topology is rebalanced to use 5 instead of 2 workers. This
> assumes, that 3 additional worker slots are available on the cluster!?
>
> 3. I read the Storm @ Twitter paper where they write 'A task is strictly
> bound to an executor because that assignment is currently static.' This
> is outdated, right? How else would the rebalancing work?
>
> 4. An executor can only execute tasks of the same component. These tasks
> are executed in serial. Right?
>
> Thanks,
> Johannes