Storm Scheduler & load balancing

[An Tran <tr...@gmail.com>]

Hello all,

I have currently have a small Storm cluster (3 nodes) each can accept 20
slots.
I have 5 topologies submitted to storm cluster currently and have set
parallelism_hits & num tasks on each bolt.

What I am finding out is that it looks like all of my bolts in the 5
topologies are running on a single machine.  Ideally if i se the number of
tasks/parallisum to 9 i like to see  3 instance of the bolts on each
machine.  Is this possible?

RE: Storm Scheduler & load balancing

[Simon Cooper <si...@featurespace.co.uk>]

There is also a bug in storm 0.9.2 where the scheduler does not consider multiple topologies when scheduling. So it will put all the topologies on the first supervisor. This is fixed in 0.9.3.

As a workaround, you can specify specific topologies to run on dedicated machines using the isolation scheduler (https://storm.apache.org/2013/01/11/storm082-released.html), or change the topology.workers property on each topology to the number of supervisors in your cluster. This will spread out all topologies evenly across the cluster.

From: Nathan Leung [mailto:ncleung@gmail.com]
Sent: 07 November 2014 14:32
To: user
Subject: Re: Storm Scheduler & load balancing


If your topology is only configured to use one worker then everything would go on one process on one machine.
On Nov 7, 2014 8:43 AM, "An Tran" <tr...@gmail.com>> wrote:
Hello all,

I have currently have a small Storm cluster (3 nodes) each can accept 20 slots.
I have 5 topologies submitted to storm cluster currently and have set parallelism_hits & num tasks on each bolt.

What I am finding out is that it looks like all of my bolts in the 5 topologies are running on a single machine.  Ideally if i se the number of tasks/parallisum to 9 i like to see  3 instance of the bolts on each machine.  Is this possible?

Re: Storm Scheduler & load balancing

[Nathan Leung <nc...@gmail.com>]

To expound, this article does a great job of explaining the difference
between tasks, executors, and workers:
http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/.
Also, 20 worker slots seems like a lot for each supervisor.  Each worker
runs its own JVM, and can contain multiple tasks and executors.  For your
example, I would reduce workers per supervisor to the default for now (4),
and set number of workers in the topology to 3.  The scheduler might not
split the workers across all the machines, but typically if nothing else is
running it would.  Then in each worker you would see 3 instances of your
bolt.

On Fri, Nov 7, 2014 at 9:32 AM, Nathan Leung <nc...@gmail.com> wrote:

> If your topology is only configured to use one worker then everything
> would go on one process on one machine.
> On Nov 7, 2014 8:43 AM, "An Tran" <tr...@gmail.com> wrote:
>
>> Hello all,
>>
>> I have currently have a small Storm cluster (3 nodes) each can accept 20
>> slots.
>> I have 5 topologies submitted to storm cluster currently and have set
>> parallelism_hits & num tasks on each bolt.
>>
>> What I am finding out is that it looks like all of my bolts in the 5
>> topologies are running on a single machine.  Ideally if i se the number of
>> tasks/parallisum to 9 i like to see  3 instance of the bolts on each
>> machine.  Is this possible?
>>
>

Re: Storm Scheduler & load balancing

[Nathan Leung <nc...@gmail.com>]

If your topology is only configured to use one worker then everything would
go on one process on one machine.
On Nov 7, 2014 8:43 AM, "An Tran" <tr...@gmail.com> wrote:

> Hello all,
>
> I have currently have a small Storm cluster (3 nodes) each can accept 20
> slots.
> I have 5 topologies submitted to storm cluster currently and have set
> parallelism_hits & num tasks on each bolt.
>
> What I am finding out is that it looks like all of my bolts in the 5
> topologies are running on a single machine.  Ideally if i se the number of
> tasks/parallisum to 9 i like to see  3 instance of the bolts on each
> machine.  Is this possible?
>