You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by Yexi Jiang <ye...@gmail.com> on 2013/11/20 17:05:30 UTC
Question about the workflow of GroomServer
Hi,
I am recently reading the code of the GroomServer and trying to figure out
the workflow. The following is my understanding, please correct me if I am
wrong.
I already know the following steps:
1. When initializing, the GroomServer will create an instructor, who will
buffer the incoming directive instance (a wrapper of action that can
launch, recover, or kill a task?).
2. The instance will be handled by the DispatchTasksHandler.
3. If the action is launched, then a couple of things will be done in the
order: DispatchTasksHandler.startNewTask -> GroomServer.localizeJob ->
GroomServer.launchTaskForJob -> TaskInProgress.launchTask .
I haven't figure out why a couple of distinct BSPJob instances are created
in these steps.
4. A TaskRunner will be created to run the task. Further, a BSPChildRunner
instance will be created. When the BSPChildRunner instance (a thread) been
scheduled and executed, it will create a process to execute
the BSPPeerChild.
5. BSPChildRunner will create a BSPPeerImpl and it will be executed by a
BSPTask (BSPTask.run -> BSPTask.runBSP).
6. BSPPeerImpl will execute its setup, bsp, and cleanup method, which will
be implemented by the end user.
Regards,
Yexi
Re: Question about the workflow of GroomServer
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hi,
> 1. Some methods have empty body and have no comments in the body, e.g.
> GroomServer.startCleanupThreads, GroomServer.process.
The startCleanupThreads method is useless (I commented when task
cleanup is handled by the same thread as the heartbeats). The process
method is just overridden.
> 2. 1. GroomServer line 131-134, 143-144, 292 - 293, 877. Are the commented
> code useless?
// private BlockingQueue<GroomServerAction> tasksToCleanup = new
// LinkedBlockingQueue<GroomServerAction>();
So, 143-144 is useless.
> 2. 2. GroomServer line 171-172, 365-266, 794-795. The TODO comments are
> not clear.
// NOTE: (currentTime - tip.lastPingedTimestamp) > 6 * monitorPeriod
Sorry for my un-readable comment. I mean, "This needs to be configurable".
> 2. 3. GroomServer line 428. Does the 'FIXME' indicate the code before it
> contains potential problem?
Looks like meaningless.
On Fri, Nov 22, 2013 at 12:14 AM, Yexi Jiang <ye...@gmail.com> wrote:
> It's not the same object initialized more than one time, it's multiple
> objects with the same class (BSPJob)initialized.
>
> Take the BSPJob for instance:
> * In GroomServer.startNewTask, an object called jobConf (BSPJob, line 546)
> is initialized with the information given by action (LaunchTaskAction).
>
> * Then in GroomServer.localizeJob, an object called defaultJobConf (BSPJob,
> line 670) is created, using the information of conf (HamaConfiguration).
>
> * After that, in the same method, an object called jobConf (BSPJob, line
> 696 or line 725 based on different condition).
>
> I'm wondering is it possible to use just one object and reuse its
> information?
>
> Moreover, the Configuration/HamaConfiguration objects may also be reused.
>
>
>
> Some trivial things:
> 1. Some methods have empty body and have no comments in the body, e.g.
> GroomServer.startCleanupThreads, GroomServer.process.
> 2. Some comments are not clear. Including:
> 2. 1. GroomServer line 131-134, 143-144, 292 - 293, 877. Are the commented
> code useless?
> 2. 2. GroomServer line 171-172, 365-266, 794-795. The TODO comments are
> not clear.
> 2. 3. GroomServer line 428. Does the 'FIXME' indicate the code before it
> contains potential problem?
>
>
>
> 2013/11/21 Suraj Menon <su...@apache.org>
>
>> You have figured it out correct. Please elaborate on which object is
>> getting instantiated twice?
>>
>> -Suraj
>>
>>
>> On Wed, Nov 20, 2013 at 11:05 AM, Yexi Jiang <ye...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > I am recently reading the code of the GroomServer and trying to figure
>> out
>> > the workflow. The following is my understanding, please correct me if I
>> am
>> > wrong.
>> >
>> > I already know the following steps:
>> > 1. When initializing, the GroomServer will create an instructor, who will
>> > buffer the incoming directive instance (a wrapper of action that can
>> > launch, recover, or kill a task?).
>> > 2. The instance will be handled by the DispatchTasksHandler.
>> > 3. If the action is launched, then a couple of things will be done in the
>> > order: DispatchTasksHandler.startNewTask -> GroomServer.localizeJob ->
>> > GroomServer.launchTaskForJob -> TaskInProgress.launchTask .
>> > I haven't figure out why a couple of distinct BSPJob instances are
>> created
>> > in these steps.
>> > 4. A TaskRunner will be created to run the task. Further, a
>> BSPChildRunner
>> > instance will be created. When the BSPChildRunner instance (a thread)
>> been
>> > scheduled and executed, it will create a process to execute
>> > the BSPPeerChild.
>> > 5. BSPChildRunner will create a BSPPeerImpl and it will be executed by a
>> > BSPTask (BSPTask.run -> BSPTask.runBSP).
>> > 6. BSPPeerImpl will execute its setup, bsp, and cleanup method, which
>> will
>> > be implemented by the end user.
>> >
>> > Regards,
>> > Yexi
>> >
>>
>
>
>
> --
> ------
> Yexi Jiang,
> ECS 251, yjian004@cs.fiu.edu
> School of Computer and Information Science,
> Florida International University
> Homepage: http://users.cis.fiu.edu/~yjian004/
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Question about the workflow of GroomServer
Posted by Yexi Jiang <ye...@gmail.com>.
It's not the same object initialized more than one time, it's multiple
objects with the same class (BSPJob)initialized.
Take the BSPJob for instance:
* In GroomServer.startNewTask, an object called jobConf (BSPJob, line 546)
is initialized with the information given by action (LaunchTaskAction).
* Then in GroomServer.localizeJob, an object called defaultJobConf (BSPJob,
line 670) is created, using the information of conf (HamaConfiguration).
* After that, in the same method, an object called jobConf (BSPJob, line
696 or line 725 based on different condition).
I'm wondering is it possible to use just one object and reuse its
information?
Moreover, the Configuration/HamaConfiguration objects may also be reused.
Some trivial things:
1. Some methods have empty body and have no comments in the body, e.g.
GroomServer.startCleanupThreads, GroomServer.process.
2. Some comments are not clear. Including:
2. 1. GroomServer line 131-134, 143-144, 292 - 293, 877. Are the commented
code useless?
2. 2. GroomServer line 171-172, 365-266, 794-795. The TODO comments are
not clear.
2. 3. GroomServer line 428. Does the 'FIXME' indicate the code before it
contains potential problem?
2013/11/21 Suraj Menon <su...@apache.org>
> You have figured it out correct. Please elaborate on which object is
> getting instantiated twice?
>
> -Suraj
>
>
> On Wed, Nov 20, 2013 at 11:05 AM, Yexi Jiang <ye...@gmail.com> wrote:
>
> > Hi,
> >
> > I am recently reading the code of the GroomServer and trying to figure
> out
> > the workflow. The following is my understanding, please correct me if I
> am
> > wrong.
> >
> > I already know the following steps:
> > 1. When initializing, the GroomServer will create an instructor, who will
> > buffer the incoming directive instance (a wrapper of action that can
> > launch, recover, or kill a task?).
> > 2. The instance will be handled by the DispatchTasksHandler.
> > 3. If the action is launched, then a couple of things will be done in the
> > order: DispatchTasksHandler.startNewTask -> GroomServer.localizeJob ->
> > GroomServer.launchTaskForJob -> TaskInProgress.launchTask .
> > I haven't figure out why a couple of distinct BSPJob instances are
> created
> > in these steps.
> > 4. A TaskRunner will be created to run the task. Further, a
> BSPChildRunner
> > instance will be created. When the BSPChildRunner instance (a thread)
> been
> > scheduled and executed, it will create a process to execute
> > the BSPPeerChild.
> > 5. BSPChildRunner will create a BSPPeerImpl and it will be executed by a
> > BSPTask (BSPTask.run -> BSPTask.runBSP).
> > 6. BSPPeerImpl will execute its setup, bsp, and cleanup method, which
> will
> > be implemented by the end user.
> >
> > Regards,
> > Yexi
> >
>
--
------
Yexi Jiang,
ECS 251, yjian004@cs.fiu.edu
School of Computer and Information Science,
Florida International University
Homepage: http://users.cis.fiu.edu/~yjian004/
Re: Question about the workflow of GroomServer
Posted by Suraj Menon <su...@apache.org>.
You have figured it out correct. Please elaborate on which object is
getting instantiated twice?
-Suraj
On Wed, Nov 20, 2013 at 11:05 AM, Yexi Jiang <ye...@gmail.com> wrote:
> Hi,
>
> I am recently reading the code of the GroomServer and trying to figure out
> the workflow. The following is my understanding, please correct me if I am
> wrong.
>
> I already know the following steps:
> 1. When initializing, the GroomServer will create an instructor, who will
> buffer the incoming directive instance (a wrapper of action that can
> launch, recover, or kill a task?).
> 2. The instance will be handled by the DispatchTasksHandler.
> 3. If the action is launched, then a couple of things will be done in the
> order: DispatchTasksHandler.startNewTask -> GroomServer.localizeJob ->
> GroomServer.launchTaskForJob -> TaskInProgress.launchTask .
> I haven't figure out why a couple of distinct BSPJob instances are created
> in these steps.
> 4. A TaskRunner will be created to run the task. Further, a BSPChildRunner
> instance will be created. When the BSPChildRunner instance (a thread) been
> scheduled and executed, it will create a process to execute
> the BSPPeerChild.
> 5. BSPChildRunner will create a BSPPeerImpl and it will be executed by a
> BSPTask (BSPTask.run -> BSPTask.runBSP).
> 6. BSPPeerImpl will execute its setup, bsp, and cleanup method, which will
> be implemented by the end user.
>
> Regards,
> Yexi
>