You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by "Yu, Tao" <Ta...@aware.com> on 2014/08/22 16:49:23 UTC
Create multiple supervisors on same node
Hi all,
Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
Any help is appreciated!
thanks,
-Tao
Re: Create multiple supervisors on same node
Posted by Amila Suriarachchi <am...@gmail.com>.
Instead of increasing supervisors, isn't it possible to increase the works
per supervisor. By default storm creates 4 workers per supervisors. I think
you can override this in storm.ymal file.
thanks,
Amila.
On Fri, Aug 22, 2014 at 8:49 AM, Yu, Tao <Ta...@aware.com> wrote:
> Hi all,
>
>
>
> Anyone knows what's the requirement to generate multiple supervisors on
> the same node (for same topology)? I can create the 1st supervisor, then I
> update the "supervisor.slots.ports" to different ports, and tried to create
> the 2nd supervisor on same node, it ends up creating a new supervisor but
> with same supervisor ID as the 1st one, so it still only has one supervisor
> on that node and storm UI shows 1 supervisor as well. Any suggestion on
> how to create the 2nd, 3rd supervisor on the same node?
>
>
>
> Any help is appreciated!
>
>
>
> thanks,
>
> -Tao
>
--
Amila Suriarachchi
blog: http://amilachinthaka.blogspot.com/
RE: Create multiple supervisors on same node
Posted by "Yu, Tao" <Ta...@aware.com>.
Thanks Derek, that makes sense for me!
Br,
-Tao
-----Original Message-----
From: Derek Dagit [mailto:derekd@yahoo-inc.com]
Sent: Friday, August 22, 2014 3:34 PM
To: user@storm.incubator.apache.org
Subject: Re: Create multiple supervisors on same node
> I also tried another scenario: instead of copying the entire storm home directory, I only use one storm home, but different storm-local dir and ports, which both are specified in storm.yaml, I can still create multiple supervisors. (Of course, every time before I start a new supervisor, I have to update the storm.yaml for different storm-local dir and ports).
You will have two supervisors writing to the same log.
I recommend creating two distinct storm home directories unless you have a good reason to have them shared. I think the code assumes it is the only supervisor writing in storm home.
--
Derek
On 8/22/14, 14:08, Yu, Tao wrote:
> Thanks Harsha!
>
> Just cleaned zookeeper data (stop and re-start zookeeper) and tried again, now I can create multiple supervisors successfully!
>
> I also tried another scenario: instead of copying the entire storm home directory, I only use one storm home, but different storm-local dir and ports, which both are specified in storm.yaml, I can still create multiple supervisors. (Of course, every time before I start a new supervisor, I have to update the storm.yaml for different storm-local dir and ports).
>
> So my new questions are:
>
> 1) what's the best approach to create multiple supervisors on same node:
>
> a) each supervisor has it's own storm home directory; Or
> b) all supervisors have common storm home directory.
>
> in both approaches, supervisors have its own storm-local dir and ports.
>
> 2) when start supervisor, can we specify storm to use custom configuration (.yaml)? For example, like:
>
> $bin/storm supervisor --config conf/myConfig.yaml
>
> Seems like storm will always use conf/storm.yaml, and I do not see any document mentions about specifying custom config file.
>
> Thanks,
> -Tao
>
> -----Original Message-----
> From: Harsha [mailto:storm@harsha.io]
> Sent: Friday, August 22, 2014 12:57 PM
> To: user@storm.incubator.apache.org
> Subject: Re: Create multiple supervisors on same node
>
> Tao,
> I tried the above steps I am able to run two supervisors on the
> same node. Did you check the logs for supervisor under storm2. If
> it didn't created a local_dir/storm dir than your supervisor
> daemon might not be running. check for logs if there are any
> errors.
> -Harsha
>
> On Fri, Aug 22, 2014, at 09:20 AM, Yu, Tao wrote:
>> Thanks Harsha!
>>
>> I tried your way, and here is what I have (major parts) in my storm.yaml:
>>
>> storm.local.dir: "/opt/grid/tao/storm/storm-0.8.2/local_data/storm"
>> supervisor.slots.ports:
>> - 6700
>> - 6701
>>
>> 1) I created the 1st supervisor, and I can see specified sub-folder
>> "local_data/storm/supervisor" was created under"
>> opt/grid/tao/storm/storm-0.8.2". That's OK!
>>
>> 2) then I copied the entire "storm-0.8.2" folder to a new "storm2"
>> ("/opt/grid/tao/storm/storm2")
>>
>> 3) delete the sub-folder "local_data" under "storm2"
>>
>> 4) updated the storm.yaml under "storm2" with below change:
>>
>> storm.local.dir: "/opt/grid/tao/storm/storm2/local_data/storm"
>> supervisor.slots.ports:
>> - 8700
>> - 8701
>>
>> 5) under "storm2", create a new supervisor.
>>
>> Then the new supervisor still has the 1st supervisor's ID. And under
>> "storm2", the sub-folder "local_data/storm" was not created.
>>
>> Does storm still use the 1st storm home directory
>> ("storm/storm-0.8.2") "local_data" folder?
>>
>> Thanks,
>> -Tao
>>
>> -----Original Message-----
>> From: Harsha [mailto:storm@harsha.io]
>> Sent: Friday, August 22, 2014 11:28 AM
>> To: user@storm.incubator.apache.org
>> Subject: Re: Create multiple supervisors on same node
>>
>> Tao,
>> you need to delete the storm-local dir under your copied over storm
>> dir ( "storm2"). Otherwise it will still pick up the same
>> supervisor-id.
>> -Harsha
>>
>> On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote:
>>> Thanks Derek!
>>>
>>> I tried your suggestion, copied the entire storm home directory
>>> (which, in my case, is "storm-0.8.2") to a new directory "storm2",
>>> then in "storm2" directory, I changed the conf/storm.yaml with
>>> different ports, and tried to create a new supervisor. Still, got
>>> the same supervisor ID as the 1st one (which I created from "storm-0.8.2" directory).
>>>
>>> Did I do anything incorrectly?
>>>
>>> -Tao
>>>
>>> -----Original Message-----
>>> From: Derek Dagit [mailto:derekd@yahoo-inc.com]
>>> Sent: Friday, August 22, 2014 11:01 AM
>>> To: user@storm.incubator.apache.org
>>> Subject: Re: Create multiple supervisors on same node
>>>
>>> The two supervisors are sharing the same state, and that is how they
>>> get the same randomly-generated ID.
>>>
>>> If I recall correctly, the default state directory is created in the
>>> current working directory of the process, so that is whatever
>>> directory you happen to be in when you start the supervisor.
>>>
>>> I think probably a good thing to do is copy the entire storm home
>>> directory, change the storm.yaml in the copy to be configured with
>>> different ports as you tried, and make sure to cd into the
>>> appropriate directory when you launch the supervisor.
>>>
>>> --
>>> Derek
>>>
>>> On 8/22/14, 9:49, Yu, Tao wrote:
>>>> Hi all,
>>>>
>>>> Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
>>>>
>>>> Any help is appreciated!
>>>>
>>>> thanks,
>>>> -Tao
>>>>
Re: Create multiple supervisors on same node
Posted by Derek Dagit <de...@yahoo-inc.com>.
> I also tried another scenario: instead of copying the entire storm home directory, I only use one storm home, but different storm-local dir and ports, which both are specified in storm.yaml, I can still create multiple supervisors. (Of course, every time before I start a new supervisor, I have to update the storm.yaml for different storm-local dir and ports).
You will have two supervisors writing to the same log.
I recommend creating two distinct storm home directories unless you have a good reason to have them shared. I think the code assumes it is the only supervisor writing in storm home.
--
Derek
On 8/22/14, 14:08, Yu, Tao wrote:
> Thanks Harsha!
>
> Just cleaned zookeeper data (stop and re-start zookeeper) and tried again, now I can create multiple supervisors successfully!
>
> I also tried another scenario: instead of copying the entire storm home directory, I only use one storm home, but different storm-local dir and ports, which both are specified in storm.yaml, I can still create multiple supervisors. (Of course, every time before I start a new supervisor, I have to update the storm.yaml for different storm-local dir and ports).
>
> So my new questions are:
>
> 1) what's the best approach to create multiple supervisors on same node:
>
> a) each supervisor has it's own storm home directory; Or
> b) all supervisors have common storm home directory.
>
> in both approaches, supervisors have its own storm-local dir and ports.
>
> 2) when start supervisor, can we specify storm to use custom configuration (.yaml)? For example, like:
>
> $bin/storm supervisor --config conf/myConfig.yaml
>
> Seems like storm will always use conf/storm.yaml, and I do not see any document mentions about specifying custom config file.
>
> Thanks,
> -Tao
>
> -----Original Message-----
> From: Harsha [mailto:storm@harsha.io]
> Sent: Friday, August 22, 2014 12:57 PM
> To: user@storm.incubator.apache.org
> Subject: Re: Create multiple supervisors on same node
>
> Tao,
> I tried the above steps I am able to run two supervisors on the
> same node. Did you check the logs for supervisor under storm2. If
> it didn't created a local_dir/storm dir than your supervisor
> daemon might not be running. check for logs if there are any
> errors.
> -Harsha
>
> On Fri, Aug 22, 2014, at 09:20 AM, Yu, Tao wrote:
>> Thanks Harsha!
>>
>> I tried your way, and here is what I have (major parts) in my storm.yaml:
>>
>> storm.local.dir: "/opt/grid/tao/storm/storm-0.8.2/local_data/storm"
>> supervisor.slots.ports:
>> - 6700
>> - 6701
>>
>> 1) I created the 1st supervisor, and I can see specified sub-folder
>> "local_data/storm/supervisor" was created under"
>> opt/grid/tao/storm/storm-0.8.2". That's OK!
>>
>> 2) then I copied the entire "storm-0.8.2" folder to a new "storm2"
>> ("/opt/grid/tao/storm/storm2")
>>
>> 3) delete the sub-folder "local_data" under "storm2"
>>
>> 4) updated the storm.yaml under "storm2" with below change:
>>
>> storm.local.dir: "/opt/grid/tao/storm/storm2/local_data/storm"
>> supervisor.slots.ports:
>> - 8700
>> - 8701
>>
>> 5) under "storm2", create a new supervisor.
>>
>> Then the new supervisor still has the 1st supervisor's ID. And under
>> "storm2", the sub-folder "local_data/storm" was not created.
>>
>> Does storm still use the 1st storm home directory
>> ("storm/storm-0.8.2") "local_data" folder?
>>
>> Thanks,
>> -Tao
>>
>> -----Original Message-----
>> From: Harsha [mailto:storm@harsha.io]
>> Sent: Friday, August 22, 2014 11:28 AM
>> To: user@storm.incubator.apache.org
>> Subject: Re: Create multiple supervisors on same node
>>
>> Tao,
>> you need to delete the storm-local dir under your copied over storm
>> dir ( "storm2"). Otherwise it will still pick up the same
>> supervisor-id.
>> -Harsha
>>
>> On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote:
>>> Thanks Derek!
>>>
>>> I tried your suggestion, copied the entire storm home directory
>>> (which, in my case, is "storm-0.8.2") to a new directory "storm2",
>>> then in "storm2" directory, I changed the conf/storm.yaml with
>>> different ports, and tried to create a new supervisor. Still, got
>>> the same supervisor ID as the 1st one (which I created from "storm-0.8.2" directory).
>>>
>>> Did I do anything incorrectly?
>>>
>>> -Tao
>>>
>>> -----Original Message-----
>>> From: Derek Dagit [mailto:derekd@yahoo-inc.com]
>>> Sent: Friday, August 22, 2014 11:01 AM
>>> To: user@storm.incubator.apache.org
>>> Subject: Re: Create multiple supervisors on same node
>>>
>>> The two supervisors are sharing the same state, and that is how they
>>> get the same randomly-generated ID.
>>>
>>> If I recall correctly, the default state directory is created in the
>>> current working directory of the process, so that is whatever
>>> directory you happen to be in when you start the supervisor.
>>>
>>> I think probably a good thing to do is copy the entire storm home
>>> directory, change the storm.yaml in the copy to be configured with
>>> different ports as you tried, and make sure to cd into the
>>> appropriate directory when you launch the supervisor.
>>>
>>> --
>>> Derek
>>>
>>> On 8/22/14, 9:49, Yu, Tao wrote:
>>>> Hi all,
>>>>
>>>> Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
>>>>
>>>> Any help is appreciated!
>>>>
>>>> thanks,
>>>> -Tao
>>>>
RE: Create multiple supervisors on same node
Posted by "Yu, Tao" <Ta...@aware.com>.
Thanks Harsha!
Just cleaned zookeeper data (stop and re-start zookeeper) and tried again, now I can create multiple supervisors successfully!
I also tried another scenario: instead of copying the entire storm home directory, I only use one storm home, but different storm-local dir and ports, which both are specified in storm.yaml, I can still create multiple supervisors. (Of course, every time before I start a new supervisor, I have to update the storm.yaml for different storm-local dir and ports).
So my new questions are:
1) what's the best approach to create multiple supervisors on same node:
a) each supervisor has it's own storm home directory; Or
b) all supervisors have common storm home directory.
in both approaches, supervisors have its own storm-local dir and ports.
2) when start supervisor, can we specify storm to use custom configuration (.yaml)? For example, like:
$bin/storm supervisor --config conf/myConfig.yaml
Seems like storm will always use conf/storm.yaml, and I do not see any document mentions about specifying custom config file.
Thanks,
-Tao
-----Original Message-----
From: Harsha [mailto:storm@harsha.io]
Sent: Friday, August 22, 2014 12:57 PM
To: user@storm.incubator.apache.org
Subject: Re: Create multiple supervisors on same node
Tao,
I tried the above steps I am able to run two supervisors on the
same node. Did you check the logs for supervisor under storm2. If
it didn't created a local_dir/storm dir than your supervisor
daemon might not be running. check for logs if there are any
errors.
-Harsha
On Fri, Aug 22, 2014, at 09:20 AM, Yu, Tao wrote:
> Thanks Harsha!
>
> I tried your way, and here is what I have (major parts) in my storm.yaml:
>
> storm.local.dir: "/opt/grid/tao/storm/storm-0.8.2/local_data/storm"
> supervisor.slots.ports:
> - 6700
> - 6701
>
> 1) I created the 1st supervisor, and I can see specified sub-folder
> "local_data/storm/supervisor" was created under "
> opt/grid/tao/storm/storm-0.8.2". That's OK!
>
> 2) then I copied the entire "storm-0.8.2" folder to a new "storm2"
> ("/opt/grid/tao/storm/storm2")
>
> 3) delete the sub-folder "local_data" under "storm2"
>
> 4) updated the storm.yaml under "storm2" with below change:
>
> storm.local.dir: "/opt/grid/tao/storm/storm2/local_data/storm"
> supervisor.slots.ports:
> - 8700
> - 8701
>
> 5) under "storm2", create a new supervisor.
>
> Then the new supervisor still has the 1st supervisor's ID. And under
> "storm2", the sub-folder "local_data/storm" was not created.
>
> Does storm still use the 1st storm home directory
> ("storm/storm-0.8.2") "local_data" folder?
>
> Thanks,
> -Tao
>
> -----Original Message-----
> From: Harsha [mailto:storm@harsha.io]
> Sent: Friday, August 22, 2014 11:28 AM
> To: user@storm.incubator.apache.org
> Subject: Re: Create multiple supervisors on same node
>
> Tao,
> you need to delete the storm-local dir under your copied over storm
> dir ( "storm2"). Otherwise it will still pick up the same
> supervisor-id.
> -Harsha
>
> On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote:
> > Thanks Derek!
> >
> > I tried your suggestion, copied the entire storm home directory
> > (which, in my case, is "storm-0.8.2") to a new directory "storm2",
> > then in "storm2" directory, I changed the conf/storm.yaml with
> > different ports, and tried to create a new supervisor. Still, got
> > the same supervisor ID as the 1st one (which I created from "storm-0.8.2" directory).
> >
> > Did I do anything incorrectly?
> >
> > -Tao
> >
> > -----Original Message-----
> > From: Derek Dagit [mailto:derekd@yahoo-inc.com]
> > Sent: Friday, August 22, 2014 11:01 AM
> > To: user@storm.incubator.apache.org
> > Subject: Re: Create multiple supervisors on same node
> >
> > The two supervisors are sharing the same state, and that is how they
> > get the same randomly-generated ID.
> >
> > If I recall correctly, the default state directory is created in the
> > current working directory of the process, so that is whatever
> > directory you happen to be in when you start the supervisor.
> >
> > I think probably a good thing to do is copy the entire storm home
> > directory, change the storm.yaml in the copy to be configured with
> > different ports as you tried, and make sure to cd into the
> > appropriate directory when you launch the supervisor.
> >
> > --
> > Derek
> >
> > On 8/22/14, 9:49, Yu, Tao wrote:
> > > Hi all,
> > >
> > > Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
> > >
> > > Any help is appreciated!
> > >
> > > thanks,
> > > -Tao
> > >
Re: Create multiple supervisors on same node
Posted by Harsha <st...@harsha.io>.
Tao,
I tried the above steps I am able to run two supervisors on the
same node. Did you check the logs for supervisor under storm2. If
it didn't created a local_dir/storm dir than your supervisor
daemon might not be running. check for logs if there are any
errors.
-Harsha
On Fri, Aug 22, 2014, at 09:20 AM, Yu, Tao wrote:
> Thanks Harsha!
>
> I tried your way, and here is what I have (major parts) in my storm.yaml:
>
> storm.local.dir: "/opt/grid/tao/storm/storm-0.8.2/local_data/storm"
> supervisor.slots.ports:
> - 6700
> - 6701
>
> 1) I created the 1st supervisor, and I can see specified sub-folder
> "local_data/storm/supervisor" was created under "
> opt/grid/tao/storm/storm-0.8.2". That's OK!
>
> 2) then I copied the entire "storm-0.8.2" folder to a new "storm2"
> ("/opt/grid/tao/storm/storm2")
>
> 3) delete the sub-folder "local_data" under "storm2"
>
> 4) updated the storm.yaml under "storm2" with below change:
>
> storm.local.dir: "/opt/grid/tao/storm/storm2/local_data/storm"
> supervisor.slots.ports:
> - 8700
> - 8701
>
> 5) under "storm2", create a new supervisor.
>
> Then the new supervisor still has the 1st supervisor's ID. And under
> "storm2", the sub-folder "local_data/storm" was not created.
>
> Does storm still use the 1st storm home directory ("storm/storm-0.8.2")
> "local_data" folder?
>
> Thanks,
> -Tao
>
> -----Original Message-----
> From: Harsha [mailto:storm@harsha.io]
> Sent: Friday, August 22, 2014 11:28 AM
> To: user@storm.incubator.apache.org
> Subject: Re: Create multiple supervisors on same node
>
> Tao,
> you need to delete the storm-local dir under your copied over storm
> dir ( "storm2"). Otherwise it will still pick up the same
> supervisor-id.
> -Harsha
>
> On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote:
> > Thanks Derek!
> >
> > I tried your suggestion, copied the entire storm home directory
> > (which, in my case, is "storm-0.8.2") to a new directory "storm2",
> > then in "storm2" directory, I changed the conf/storm.yaml with
> > different ports, and tried to create a new supervisor. Still, got the
> > same supervisor ID as the 1st one (which I created from "storm-0.8.2" directory).
> >
> > Did I do anything incorrectly?
> >
> > -Tao
> >
> > -----Original Message-----
> > From: Derek Dagit [mailto:derekd@yahoo-inc.com]
> > Sent: Friday, August 22, 2014 11:01 AM
> > To: user@storm.incubator.apache.org
> > Subject: Re: Create multiple supervisors on same node
> >
> > The two supervisors are sharing the same state, and that is how they
> > get the same randomly-generated ID.
> >
> > If I recall correctly, the default state directory is created in the
> > current working directory of the process, so that is whatever
> > directory you happen to be in when you start the supervisor.
> >
> > I think probably a good thing to do is copy the entire storm home
> > directory, change the storm.yaml in the copy to be configured with
> > different ports as you tried, and make sure to cd into the appropriate
> > directory when you launch the supervisor.
> >
> > --
> > Derek
> >
> > On 8/22/14, 9:49, Yu, Tao wrote:
> > > Hi all,
> > >
> > > Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
> > >
> > > Any help is appreciated!
> > >
> > > thanks,
> > > -Tao
> > >
RE: Create multiple supervisors on same node
Posted by "Yu, Tao" <Ta...@aware.com>.
Thanks Harsha!
I tried your way, and here is what I have (major parts) in my storm.yaml:
storm.local.dir: "/opt/grid/tao/storm/storm-0.8.2/local_data/storm"
supervisor.slots.ports:
- 6700
- 6701
1) I created the 1st supervisor, and I can see specified sub-folder "local_data/storm/supervisor" was created under " opt/grid/tao/storm/storm-0.8.2". That's OK!
2) then I copied the entire "storm-0.8.2" folder to a new "storm2" ("/opt/grid/tao/storm/storm2")
3) delete the sub-folder "local_data" under "storm2"
4) updated the storm.yaml under "storm2" with below change:
storm.local.dir: "/opt/grid/tao/storm/storm2/local_data/storm"
supervisor.slots.ports:
- 8700
- 8701
5) under "storm2", create a new supervisor.
Then the new supervisor still has the 1st supervisor's ID. And under "storm2", the sub-folder "local_data/storm" was not created.
Does storm still use the 1st storm home directory ("storm/storm-0.8.2") "local_data" folder?
Thanks,
-Tao
-----Original Message-----
From: Harsha [mailto:storm@harsha.io]
Sent: Friday, August 22, 2014 11:28 AM
To: user@storm.incubator.apache.org
Subject: Re: Create multiple supervisors on same node
Tao,
you need to delete the storm-local dir under your copied over storm
dir ( "storm2"). Otherwise it will still pick up the same
supervisor-id.
-Harsha
On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote:
> Thanks Derek!
>
> I tried your suggestion, copied the entire storm home directory
> (which, in my case, is "storm-0.8.2") to a new directory "storm2",
> then in "storm2" directory, I changed the conf/storm.yaml with
> different ports, and tried to create a new supervisor. Still, got the
> same supervisor ID as the 1st one (which I created from "storm-0.8.2" directory).
>
> Did I do anything incorrectly?
>
> -Tao
>
> -----Original Message-----
> From: Derek Dagit [mailto:derekd@yahoo-inc.com]
> Sent: Friday, August 22, 2014 11:01 AM
> To: user@storm.incubator.apache.org
> Subject: Re: Create multiple supervisors on same node
>
> The two supervisors are sharing the same state, and that is how they
> get the same randomly-generated ID.
>
> If I recall correctly, the default state directory is created in the
> current working directory of the process, so that is whatever
> directory you happen to be in when you start the supervisor.
>
> I think probably a good thing to do is copy the entire storm home
> directory, change the storm.yaml in the copy to be configured with
> different ports as you tried, and make sure to cd into the appropriate
> directory when you launch the supervisor.
>
> --
> Derek
>
> On 8/22/14, 9:49, Yu, Tao wrote:
> > Hi all,
> >
> > Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
> >
> > Any help is appreciated!
> >
> > thanks,
> > -Tao
> >
Re: Create multiple supervisors on same node
Posted by Harsha <st...@harsha.io>.
Tao,
you need to delete the storm-local dir under your copied over storm
dir ( "storm2"). Otherwise it will still pick up the same
supervisor-id.
-Harsha
On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote:
> Thanks Derek!
>
> I tried your suggestion, copied the entire storm home directory (which,
> in my case, is "storm-0.8.2") to a new directory "storm2", then in
> "storm2" directory, I changed the conf/storm.yaml with different ports,
> and tried to create a new supervisor. Still, got the same supervisor ID
> as the 1st one (which I created from "storm-0.8.2" directory).
>
> Did I do anything incorrectly?
>
> -Tao
>
> -----Original Message-----
> From: Derek Dagit [mailto:derekd@yahoo-inc.com]
> Sent: Friday, August 22, 2014 11:01 AM
> To: user@storm.incubator.apache.org
> Subject: Re: Create multiple supervisors on same node
>
> The two supervisors are sharing the same state, and that is how they get
> the same randomly-generated ID.
>
> If I recall correctly, the default state directory is created in the
> current working directory of the process, so that is whatever directory
> you happen to be in when you start the supervisor.
>
> I think probably a good thing to do is copy the entire storm home
> directory, change the storm.yaml in the copy to be configured with
> different ports as you tried, and make sure to cd into the appropriate
> directory when you launch the supervisor.
>
> --
> Derek
>
> On 8/22/14, 9:49, Yu, Tao wrote:
> > Hi all,
> >
> > Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
> >
> > Any help is appreciated!
> >
> > thanks,
> > -Tao
> >
RE: Create multiple supervisors on same node
Posted by "Yu, Tao" <Ta...@aware.com>.
Thanks Derek!
I tried your suggestion, copied the entire storm home directory (which, in my case, is "storm-0.8.2") to a new directory "storm2", then in "storm2" directory, I changed the conf/storm.yaml with different ports, and tried to create a new supervisor. Still, got the same supervisor ID as the 1st one (which I created from "storm-0.8.2" directory).
Did I do anything incorrectly?
-Tao
-----Original Message-----
From: Derek Dagit [mailto:derekd@yahoo-inc.com]
Sent: Friday, August 22, 2014 11:01 AM
To: user@storm.incubator.apache.org
Subject: Re: Create multiple supervisors on same node
The two supervisors are sharing the same state, and that is how they get the same randomly-generated ID.
If I recall correctly, the default state directory is created in the current working directory of the process, so that is whatever directory you happen to be in when you start the supervisor.
I think probably a good thing to do is copy the entire storm home directory, change the storm.yaml in the copy to be configured with different ports as you tried, and make sure to cd into the appropriate directory when you launch the supervisor.
--
Derek
On 8/22/14, 9:49, Yu, Tao wrote:
> Hi all,
>
> Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
>
> Any help is appreciated!
>
> thanks,
> -Tao
>
Re: Create multiple supervisors on same node
Posted by Derek Dagit <de...@yahoo-inc.com>.
The two supervisors are sharing the same state, and that is how they get the same randomly-generated ID.
If I recall correctly, the default state directory is created in the current working directory of the process, so that is whatever directory you happen to be in when you start the supervisor.
I think probably a good thing to do is copy the entire storm home directory, change the storm.yaml in the copy to be configured with different ports as you tried, and make sure to cd into the appropriate directory when you launch the supervisor.
--
Derek
On 8/22/14, 9:49, Yu, Tao wrote:
> Hi all,
>
> Anyone knows what's the requirement to generate multiple supervisors on the same node (for same topology)? I can create the 1st supervisor, then I update the "supervisor.slots.ports" to different ports, and tried to create the 2nd supervisor on same node, it ends up creating a new supervisor but with same supervisor ID as the 1st one, so it still only has one supervisor on that node and storm UI shows 1 supervisor as well. Any suggestion on how to create the 2nd, 3rd supervisor on the same node?
>
> Any help is appreciated!
>
> thanks,
> -Tao
>