You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2008/05/23 13:29:55 UTC
[jira] Created: (HADOOP-3438) NPE if job tracker started and system
property hadoop.log.dir is not set
NPE if job tracker started and system property hadoop.log.dir is not set
------------------------------------------------------------------------
Key: HADOOP-3438
URL: https://issues.apache.org/jira/browse/HADOOP-3438
Project: Hadoop Core
Issue Type: Bug
Components: metrics
Affects Versions: 0.18.0
Environment: amd64 ubuntu, jrockit 1.6
Reporter: Steve Loughran
This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3438) NPE if job tracker started and system
property hadoop.log.dir is not set
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-3438:
----------------------------------------
Fix Version/s: 0.21.0
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
> Fix For: 0.21.0
>
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599333#action_12599333 ]
Steve Loughran commented on HADOOP-3438:
----------------------------------------
Stack trace
Caused by: java.lang.NullPointerException
at java.io.File.(File.java:222)
at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:121)
at org.apache.hadoop.mapred.JobTracker.(JobTracker.java:672)
//my code starts here
at org.apache.hadoop.mapred.ExtJobTracker.(ExtJobTracker.java:41)
at org.smartfrog.services.hadoop.components.tracker.JobTrackerImpl.sfStart(JobTrackerImpl.java:53)
at org.smartfrog.sfcore.compound.CompoundImpl.sfStartChildren(CompoundImpl.java:659)
at org.smartfrog.sfcore.compound.CompoundImpl.sfStart(CompoundImpl.java:632)
at org.smartfrog.services.assertions.TestCompoundImpl.sfStart(TestCompoundImpl.java:249)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599380#action_12599380 ]
Steve Loughran commented on HADOOP-3438:
----------------------------------------
It's not that I want to disable logging - I do have logging enabled in my configuration - its that the system property "hadoop.log.dir" needs to be set regardless of whether or not hadoop.job.history.location is defined in the config file.
if the property is null, then new File(System.getProperty("hadoop.log.dir")) fails. This object is created before conf.get() is called to look for my configuration-defined settings.
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599336#action_12599336 ]
Steve Loughran commented on HADOOP-3438:
----------------------------------------
Root cause is in JobHistory.init. It tries to get the configuration option fo the job history, but to give it a useful default the system property "hadoop.job.history.location" is used. If that property is not set, ops get a support call.
public static boolean init(JobConf conf, String hostname){
try {
LOG_DIR = conf.get("hadoop.job.history.location" ,
"file:///" + new File(
System.getProperty("hadoop.log.dir")).getAbsolutePath()
+ File.separator + "history");
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608778#action_12608778 ]
Steve Loughran commented on HADOOP-3438:
----------------------------------------
I should add that this makes running under the debugger trickier too. This property should only be needed if no log dir is set in the configuration.
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599384#action_12599384 ]
Devaraj Das commented on HADOOP-3438:
-------------------------------------
Are you running the JobTracker without using bin/hadoop?
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599387#action_12599387 ]
Steve Loughran commented on HADOOP-3438:
----------------------------------------
Thats right, and I was doing so with the 0.16.x branch too; moving up to SVN_HEAD broke things.
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12666677#action_12666677 ]
Konstantin Shvachko commented on HADOOP-3438:
---------------------------------------------
I have the same problem as Steve.
I think JobHistory should not use System.getProperty().
If it has to then at least use {{getProperty(String key, String def)}} instead.
NPE should not happen anyways.
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
> Fix For: 0.21.0
>
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3438) NPE if job tracker started and
system property hadoop.log.dir is not set
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599370#action_12599370 ]
Devaraj Das commented on HADOOP-3438:
-------------------------------------
Do you really want to disable logging in the JT? There are other things like the link in the Job webUi to do with jobconf won't work as well. To maintain correctness, we should check for null in all those places.
> NPE if job tracker started and system property hadoop.log.dir is not set
> ------------------------------------------------------------------------
>
> Key: HADOOP-3438
> URL: https://issues.apache.org/jira/browse/HADOOP-3438
> Project: Hadoop Core
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.18.0
> Environment: amd64 ubuntu, jrockit 1.6
> Reporter: Steve Loughran
>
> This is a regression. If the system property "hadoop.log.dir" is not set, the job tracker NPEs rather than starts up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.