You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by jeff saremi <je...@hotmail.com> on 2014/07/06 17:20:10 UTC
The role of supervisor in Storm
I posted another note earlier asking for hints and links on in-depth architecture documentation. I didn't get any responses. Which probably means I am the only one with this problem.
So i'll just go ahead and ask specific questions:
What is the role of the supervisor? I know that it starts and stops the worker processes. But does it monitor them? how? Who monitors the supervisor itself if it goes down? Launching something with "%java storm something" is referred to as launching the item (nimbus) under supervision. Can anything be launched like that? Is Supervisable an interface? or a protocol?
I tried reading the supervisor's code but it looked like hieroglyphs to me.
thanks
Jeff
RE: The role of supervisor in Storm
Posted by jeff saremi <je...@hotmail.com>.
Thanks Bobby. good description. I think most of confusion was around the Storm supervisor versus a generic process supervisor. The similarity of names had thrown me off the course completely.
... and thanks for the cheat sheet :)Jeff
From: evans@yahoo-inc.com
To: user@storm.incubator.apache.org; dev@storm.incubator.apache.org
Subject: Re: The role of supervisor in Storm
Date: Mon, 7 Jul 2014 16:30:49 +0000
The supervisor will launch, kill and monitor worker processes. The monitoring is done through heartbeat files that the worker process writes out periodically and the supervisor reads in. The supervisor itself is expected to be run under supervision.
This means that you need an external process that will restart the supervisor if it goes down for any reason. We use daemontools (http://en.wikipedia.org/wiki/Daemontools) but there are several other
tools that work to do the same sort of thing.
As for the hieroglyphs http://clojure.org/cheatsheet might be an OK rosetta stone for you.
- Bobby
From: jeff saremi <je...@hotmail.com>
Reply-To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>
Date: Sunday, July 6, 2014 at 10:20 AM
To: "dev@storm.incubator.apache.org" <de...@storm.incubator.apache.org>, "user@storm.incubator.apache.org"
<us...@storm.incubator.apache.org>
Subject: The role of supervisor in Storm
I posted another note earlier asking for hints and links on in-depth architecture documentation. I didn't get any responses. Which probably means I am the only one with this problem.
So i'll just go ahead and ask specific questions:
What is the role of the supervisor? I know that it starts and stops the worker processes. But does it monitor them? how? Who monitors the supervisor itself if it goes down? Launching something with "%java storm something" is referred to as launching the item
(nimbus) under supervision. Can anything be launched like that? Is Supervisable an interface? or a protocol?
I tried reading the supervisor's code but it looked like hieroglyphs to me.
thanks
Jeff
RE: The role of supervisor in Storm
Posted by jeff saremi <je...@hotmail.com>.
Thanks Bobby. good description. I think most of confusion was around the Storm supervisor versus a generic process supervisor. The similarity of names had thrown me off the course completely.
... and thanks for the cheat sheet :)Jeff
From: evans@yahoo-inc.com
To: user@storm.incubator.apache.org; dev@storm.incubator.apache.org
Subject: Re: The role of supervisor in Storm
Date: Mon, 7 Jul 2014 16:30:49 +0000
The supervisor will launch, kill and monitor worker processes. The monitoring is done through heartbeat files that the worker process writes out periodically and the supervisor reads in. The supervisor itself is expected to be run under supervision.
This means that you need an external process that will restart the supervisor if it goes down for any reason. We use daemontools (http://en.wikipedia.org/wiki/Daemontools) but there are several other
tools that work to do the same sort of thing.
As for the hieroglyphs http://clojure.org/cheatsheet might be an OK rosetta stone for you.
- Bobby
From: jeff saremi <je...@hotmail.com>
Reply-To: "user@storm.incubator.apache.org" <us...@storm.incubator.apache.org>
Date: Sunday, July 6, 2014 at 10:20 AM
To: "dev@storm.incubator.apache.org" <de...@storm.incubator.apache.org>, "user@storm.incubator.apache.org"
<us...@storm.incubator.apache.org>
Subject: The role of supervisor in Storm
I posted another note earlier asking for hints and links on in-depth architecture documentation. I didn't get any responses. Which probably means I am the only one with this problem.
So i'll just go ahead and ask specific questions:
What is the role of the supervisor? I know that it starts and stops the worker processes. But does it monitor them? how? Who monitors the supervisor itself if it goes down? Launching something with "%java storm something" is referred to as launching the item
(nimbus) under supervision. Can anything be launched like that? Is Supervisable an interface? or a protocol?
I tried reading the supervisor's code but it looked like hieroglyphs to me.
thanks
Jeff
Re: The role of supervisor in Storm
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
The supervisor will launch, kill and monitor worker processes. The monitoring is done through heartbeat files that the worker process writes out periodically and the supervisor reads in. The supervisor itself is expected to be run under supervision. This means that you need an external process that will restart the supervisor if it goes down for any reason. We use daemontools (http://en.wikipedia.org/wiki/Daemontools) but there are several other tools that work to do the same sort of thing.
As for the hieroglyphs http://clojure.org/cheatsheet might be an OK rosetta stone for you.
- Bobby
From: jeff saremi <je...@hotmail.com>>
Reply-To: "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Date: Sunday, July 6, 2014 at 10:20 AM
To: "dev@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <de...@storm.incubator.apache.org>>, "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Subject: The role of supervisor in Storm
I posted another note earlier asking for hints and links on in-depth architecture documentation. I didn't get any responses. Which probably means I am the only one with this problem.
So i'll just go ahead and ask specific questions:
What is the role of the supervisor? I know that it starts and stops the worker processes. But does it monitor them? how? Who monitors the supervisor itself if it goes down? Launching something with "%java storm something" is referred to as launching the item (nimbus) under supervision. Can anything be launched like that? Is Supervisable an interface? or a protocol?
I tried reading the supervisor's code but it looked like hieroglyphs to me.
thanks
Jeff
Re: The role of supervisor in Storm
Posted by Bobby Evans <ev...@yahoo-inc.com>.
The supervisor will launch, kill and monitor worker processes. The monitoring is done through heartbeat files that the worker process writes out periodically and the supervisor reads in. The supervisor itself is expected to be run under supervision. This means that you need an external process that will restart the supervisor if it goes down for any reason. We use daemontools (http://en.wikipedia.org/wiki/Daemontools) but there are several other tools that work to do the same sort of thing.
As for the hieroglyphs http://clojure.org/cheatsheet might be an OK rosetta stone for you.
- Bobby
From: jeff saremi <je...@hotmail.com>>
Reply-To: "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Date: Sunday, July 6, 2014 at 10:20 AM
To: "dev@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <de...@storm.incubator.apache.org>>, "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Subject: The role of supervisor in Storm
I posted another note earlier asking for hints and links on in-depth architecture documentation. I didn't get any responses. Which probably means I am the only one with this problem.
So i'll just go ahead and ask specific questions:
What is the role of the supervisor? I know that it starts and stops the worker processes. But does it monitor them? how? Who monitors the supervisor itself if it goes down? Launching something with "%java storm something" is referred to as launching the item (nimbus) under supervision. Can anything be launched like that? Is Supervisable an interface? or a protocol?
I tried reading the supervisor's code but it looked like hieroglyphs to me.
thanks
Jeff