You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Created) (JIRA)" <ji...@apache.org> on 2012/02/21 13:23:34 UTC
[jira] [Created] (HADOOP-8096) add single point where System.exit()
is called for better handling in containers
add single point where System.exit() is called for better handling in containers
--------------------------------------------------------------------------------
Key: HADOOP-8096
URL: https://issues.apache.org/jira/browse/HADOOP-8096
Project: Hadoop Common
Issue Type: New Feature
Components: util
Affects Versions: 0.24.0
Reporter: Steve Loughran
Fix For: 0.24.0
with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8096) add single point where System.exit()
is called for better handling in containers
Posted by "Csaba Miklos (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Csaba Miklos updated HADOOP-8096:
---------------------------------
Attachment: HADOOP-8096.patch
Exiter class added. (Steve's code moved from Jira to a java file)
Maybe I am misunderstood something (maybe because I am a newbie in Hadoop), but you wrote to the issue that this Exiter should call from container.
So I need to replace every System.exit call, or just some of them?
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409920#comment-13409920 ]
Aaron T. Myers commented on HADOOP-8096:
----------------------------------------
Looks like this work might overlap with some of the work that's going on in HDFS-3582, though that work is so far HDFS-only.
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Steve Loughran (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231051#comment-13231051 ]
Steve Loughran commented on HADOOP-8096:
----------------------------------------
The place where it would be useful is those routines that see themselves in a mess and go {{LOG.error(text, e); System.exit(-1)}}. In classes in the main thread translating this to an exception and raising it will intercept the problem and can trigger a more graceful shutdown in an OSGi container.
What is tricky is threads who make the same call; translate this to an exception and the thread goes away, but the exception isn't reported up and the code keeps going, only now some thread has disappeared. In this situation there's a need to report the problem to the container.
The OSGi experts need to be involved here, as the only thing I can think of is having the {{catch(SecurityException se)}} handler look for some JVM property giving the name of a class to load and reflect into that for a method like {{systemExitCalled(...)}}. Which puts you straight into the world of OSGi classloaders, as this handler would presumably be something in the container.
Of course, that could be feature creep once the method is in - and system behaviour will be the same as normal when not run under a security handler.
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407928#comment-13407928 ]
Guillaume Nodet commented on HADOOP-8096:
-----------------------------------------
Fwiw, I've been able to run hadoop in osgi (I'm working on patches right now) and I haven't hit any issues with System.exit being called.
I think most of the calls are in Main classes (tools or main runners) and can be avoided by controlling the configuration in advance.
For those cases, I think it would be better to have the code throw an exception and catch it in the respective main() methods and call System.exit() there.
In OSGi, the main() methods would not be called, to the OSGi layer would have a way to intercept the problems gracefully.
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Csaba Miklos (Work started) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-8096 started by Csaba Miklos.
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8096) add single point where System.exit()
is called for better handling in containers
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated HADOOP-8096:
----------------------------
Fix Version/s: (was: 0.24.0)
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 2.0.0-alpha
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212626#comment-13212626 ]
Suresh Srinivas commented on HADOOP-8096:
-----------------------------------------
+1 for the idea.
I think you meant int to be String. Will review this when a patch is available.
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Priority: Trivial
> Fix For: 0.24.0
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8096) add single point where System.exit()
is called for better handling in containers
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated HADOOP-8096:
----------------------------
Affects Version/s: (was: 0.24.0)
2.0.0-alpha
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 2.0.0-alpha
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
> Attachments: HADOOP-8096.patch
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Csaba Miklos (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Csaba Miklos reassigned HADOOP-8096:
------------------------------------
Assignee: Csaba Miklos
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Assignee: Csaba Miklos
> Priority: Trivial
> Fix For: 0.24.0
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8096) add single point where
System.exit() is called for better handling in containers
Posted by "Steve Loughran (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212539#comment-13212539 ]
Steve Loughran commented on HADOOP-8096:
----------------------------------------
code would look something like this:
{code}
public class Exiter {
public static Log LOG = LogFactory.getLog(Exiter.class);
public static void SystemExit(int exitCode, int reason) {
String operation = Thread.currentThread() + ": exiting with code "
+ exitCode + " reason" + reason;
LOG.info(operation);
try {
System.exit(exitCode);
} catch (SecurityException e) {
LOG.info(operation + " trapped by security manager");
throw new SecurityException(operation, e);
}
}
}
{code}
Calls of System.exit in -hdfs and -mapreduce would be replaced by calls of this with a reason string. Maybe an exception could optionally be passed down too.
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Fix For: 0.24.0
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8096) add single point where System.exit()
is called for better handling in containers
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-8096:
-----------------------------------
Priority: Trivial (was: Major)
> add single point where System.exit() is called for better handling in containers
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8096
> URL: https://issues.apache.org/jira/browse/HADOOP-8096
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.24.0
> Reporter: Steve Loughran
> Priority: Trivial
> Fix For: 0.24.0
>
>
> with plans for OSGI integration afoot in HADOOP-7977, Hadoop needs unified place where System.exit() calls. When one runs any bit of Hadoop in a containers the container will block those exits with a security manager and convert the calls into security exceptions. A single exit method would enable such exceptions to be logged, and conceivably handled slightly more gracefully (e.g. tell the daemon to die).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira