You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Shamsul Haque <sh...@corp.india.com> on 2014/08/20 12:13:23 UTC
issue in storm submit on multiple machine
Hi,
I have 9 storm topologies to deploy on 2 storm servers:
server 1: i have started nimbus + supervisor
server 2: started only supervisor
i have planned to run 5 topologies on server-1 and 4 on server-2.
But when i submit my topologies from server-2 the topology moved on
server-1 and start executing on server-1, but i think it should run on
server-2 (on which i have submitted).
Please suggest, why this behaviour?
--
Thanks and regards
Shams ul Haque
Re: issue in storm submit on multiple machine
Posted by Spico Florin <sp...@gmail.com>.
Hello!
Perhaps you need a combination of scheduling of topology isolation
https://storm.incubator.apache.org/2013/01/11/storm082-released.html and
customized scheduling
http://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/
.
Hope that these help.
Regards,
Florin
On Tue, Aug 26, 2014 at 4:07 PM, Shamsul Haque <sh...@corp.india.com>
wrote:
> Hi Ankit,
>
> I have 2 workers on each topology. I used your suggestion, but when i
> submitted jar from worker machine then it only registered that topology on
> nimbus machine(server 1) but doesn’t start. And when i ran supervisor on
> nimbus machine(server 1), then it started those topology on nimbus
> machine(server 1) only but not on worker machine(server 2). Still worker
> machine(server 2) is not in use.
>
> I tried also by restricting number of supervisor.slots.ports. First i
> started topologies on nimbus machine(server 1) and when all ports get
> assigned on nimbus machine then i started supervisor on worker
> machine(server 2) and then remaining topologies. But in this case also
> topologies moves to nimbus machine(server 1) with 0 workers assigned to
> them(as i restricted number of ports).
> I don't know why this behaviour. Does this works for you perfectly?
>
>
> Thanks and regards
> Shams ul Haque
>
>
>
> On Thursday 21 August 2014 03:12 AM, Ankit Toshniwal wrote:
>
> Your topology is configured for how many workers?
>
> Nimbus tries (best effort) to evenly distribute the workers between the
> number of machines available (depends on memory etc). So for example if you
> are submitting a topology with 20 workers and if you have 2 machines
> available then Storm will start 10 workers on each machine.
>
> In your case, once you submit your topology, even if you want your
> topology to run on server 2 at the end it depends on Nimbus how it will do
> the assignments based on the resources available.
>
> If you still want to run your topology only on server2, stop supervisor
> on server1 and then re-submit the topology or do rebalance.
>
> Hope that helps!
>
> Ankit
>
>
> On Wed, Aug 20, 2014 at 3:13 AM, Shamsul Haque <sh...@corp.india.com>
> wrote:
>
>>
>> Hi,
>>
>> I have 9 storm topologies to deploy on 2 storm servers:
>> server 1: i have started nimbus + supervisor
>> server 2: started only supervisor
>> i have planned to run 5 topologies on server-1 and 4 on server-2.
>> But when i submit my topologies from server-2 the topology moved on
>> server-1 and start executing on server-1, but i think it should run on
>> server-2 (on which i have submitted).
>>
>>
>> Please suggest, why this behaviour?
>>
>> --
>> Thanks and regards
>> Shams ul Haque
>>
>>
>
Re: issue in storm submit on multiple machine
Posted by Shamsul Haque <sh...@corp.india.com>.
Hi Ankit,
I have 2 workers on each topology. I used your suggestion, but when i
submitted jar from worker machine then it only registered that topology
on nimbus machine(server 1) but doesn’t start. And when i ran supervisor
on nimbus machine(server 1), then it started those topology on nimbus
machine(server 1) only but not on worker machine(server 2). Still worker
machine(server 2) is not in use.
I tried also by restricting number of supervisor.slots.ports. First i
started topologies on nimbus machine(server 1) and when all ports get
assigned on nimbus machine then i started supervisor on worker
machine(server 2) and then remaining topologies. But in this case also
topologies moves to nimbus machine(server 1) with 0 workers assigned to
them(as i restricted number of ports).
I don't know why this behaviour. Does this works for you perfectly?
Thanks and regards
Shams ul Haque
On Thursday 21 August 2014 03:12 AM, Ankit Toshniwal wrote:
> Your topology is configured for how many workers?
>
> Nimbus tries (best effort) to evenly distribute the workers between
> the number of machines available (depends on memory etc). So for
> example if you are submitting a topology with 20 workers and if you
> have 2 machines available then Storm will start 10 workers on each
> machine.
>
> In your case, once you submit your topology, even if you want your
> topology to run on server 2 at the end it depends on Nimbus how it
> will do the assignments based on the resources available.
>
> If you still want to run your topology only on server2, stop
> supervisor on server1 and then re-submit the topology or do rebalance.
>
> Hope that helps!
>
> Ankit
>
>
> On Wed, Aug 20, 2014 at 3:13 AM, Shamsul Haque <shamsul@corp.india.com
> <ma...@corp.india.com>> wrote:
>
>
> Hi,
>
> I have 9 storm topologies to deploy on 2 storm servers:
> server 1: i have started nimbus + supervisor
> server 2: started only supervisor
> i have planned to run 5 topologies on server-1 and 4 on server-2.
> But when i submit my topologies from server-2 the topology moved
> on server-1 and start executing on server-1, but i think it should
> run on server-2 (on which i have submitted).
>
>
> Please suggest, why this behaviour?
>
> --
> Thanks and regards
> Shams ul Haque
>
>
Re: issue in storm submit on multiple machine
Posted by Ankit Toshniwal <an...@gmail.com>.
Your topology is configured for how many workers?
Nimbus tries (best effort) to evenly distribute the workers between the
number of machines available (depends on memory etc). So for example if you
are submitting a topology with 20 workers and if you have 2 machines
available then Storm will start 10 workers on each machine.
In your case, once you submit your topology, even if you want your topology
to run on server 2 at the end it depends on Nimbus how it will do the
assignments based on the resources available.
If you still want to run your topology only on server2, stop supervisor on
server1 and then re-submit the topology or do rebalance.
Hope that helps!
Ankit
On Wed, Aug 20, 2014 at 3:13 AM, Shamsul Haque <sh...@corp.india.com>
wrote:
>
> Hi,
>
> I have 9 storm topologies to deploy on 2 storm servers:
> server 1: i have started nimbus + supervisor
> server 2: started only supervisor
> i have planned to run 5 topologies on server-1 and 4 on server-2.
> But when i submit my topologies from server-2 the topology moved on
> server-1 and start executing on server-1, but i think it should run on
> server-2 (on which i have submitted).
>
>
> Please suggest, why this behaviour?
>
> --
> Thanks and regards
> Shams ul Haque
>
>