[jira] Updated: (HADOOP-3628) Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.

["Steve Loughran (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran updated HADOOP-3628:
-----------------------------------

    Attachment: hadoop-3628.patch

This patch isnt ready to go into the code; I'm putting it up to show progress.

-ping() now builds up a status that can include a list of exceptions, so a ping failure can list everything that is wrong. One thing that is unclear is should a ping() failure trigger a transition into the FAILED state, or should the code hope that the system can recover?

-the Service class implements Closeable, and the void close() throws IOException; method of that interface -it means close me and you can throw an exception. terminate() relays to close() and logs the failure. If you want to close quietly, use terminate(); if you are OK with an exception call close(). Anything aggregating other services has to call terminate() or othewise discard the exceptions, so it does a best case shut down. 

-I've moved MockService into the core, though tools is probably better. Why out of test? Because it turns out to be invaluable for hadoop management tools to test their ability to handle failures in the service lifecycle. 

I dont think ping() and the termination logic are stable yet; they seem over complex. I'd rather Closeable() and close() everywhere, including other things that are aggregate services (MiniDFSCluster, MiniMRCluster). 

> Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-3628
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3628
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs, mapred
>    Affects Versions: 0.19.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: AbstractHadoopComponent.java, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-lifecycle.pdf, hadoop-lifecycle.sxw
>
>
> I'd like to propose we have a standard interface for hadoop components, the things that get started or stopped when you bring up a namenode. currently, some of these classes have a stop() or shutdown() method, with no standard name/interface, but no way of seeing if they are live, checking their health of shutting them down reliably. Indeed, there is a tendency for the spawned threads to not want to die; to require the entire process to be killed to stop the workers. 
> Having a standard interface would make it easier for 
>  * management tools to manage the different things
>  * monitoring the state of things
>  * subclassing
> The latter is interesting as right now TaskTracker and JobTracker start up threads in their constructor; that's very dangerous as subclasses may have their methods called before they are full initialised. Adding this interface would be the right time to clean up the startup process so that subclassing is less risky.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.