You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Nigel Daley (JIRA)" <ji...@apache.org> on 2009/03/04 05:27:56 UTC
[jira] Created: (HADOOP-5389) NullPointerException in fairshare
PoolManager
NullPointerException in fairshare PoolManager
---------------------------------------------
Key: HADOOP-5389
URL: https://issues.apache.org/jira/browse/HADOOP-5389
Project: Hadoop Core
Issue Type: Bug
Components: contrib/fair-share
Affects Versions: 0.20.0
Reporter: Nigel Daley
Found this NPE in a Hadoop 0.20.0 JobTracker log:
{code}
2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
java.lang.NullPointerException
at java.io.File.<init>(File.java:222)
at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5389) NullPointerException in fairshare
PoolManager
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678630#action_12678630 ]
Vinod K V commented on HADOOP-5389:
-----------------------------------
If mapred.fairscheduler.allocation.file is not set to anything, then this exception will be seen once in the logs, but scheduler should continue to function normally. You should not see any other side effects because of this. Nige, can you confirm this?
> NullPointerException in fairshare PoolManager
> ---------------------------------------------
>
> Key: HADOOP-5389
> URL: https://issues.apache.org/jira/browse/HADOOP-5389
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.20.0
> Reporter: Nigel Daley
> Assignee: Vinod K V
>
> Found this NPE in a Hadoop 0.20.0 JobTracker log:
> {code}
> 2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5389) NullPointerException in fairshare
PoolManager
Posted by "Matei Zaharia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683891#action_12683891 ]
Matei Zaharia commented on HADOOP-5389:
---------------------------------------
That sounds good.
As a separate issue, we should include at least a template for the config file in the conf directory (I think I had another JIRA on this).
> NullPointerException in fairshare PoolManager
> ---------------------------------------------
>
> Key: HADOOP-5389
> URL: https://issues.apache.org/jira/browse/HADOOP-5389
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.20.0
> Reporter: Nigel Daley
> Assignee: Vinod K V
>
> Found this NPE in a Hadoop 0.20.0 JobTracker log:
> {code}
> 2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5389) NullPointerException in fairshare
PoolManager
Posted by "Matei Zaharia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683693#action_12683693 ]
Matei Zaharia commented on HADOOP-5389:
---------------------------------------
When this happens, there are no minimum shares and weights configured for pools. It is as if you had an empty allocation file. If you'd prefer to just prevent the JT from running if there's no config file, we can throw a RuntimeException.
> NullPointerException in fairshare PoolManager
> ---------------------------------------------
>
> Key: HADOOP-5389
> URL: https://issues.apache.org/jira/browse/HADOOP-5389
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.20.0
> Reporter: Nigel Daley
> Assignee: Vinod K V
>
> Found this NPE in a Hadoop 0.20.0 JobTracker log:
> {code}
> 2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5389) NullPointerException in fairshare
PoolManager
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678755#action_12678755 ]
Nigel Daley commented on HADOOP-5389:
-------------------------------------
Although it's hard to assure there were no other side effects, I was able to submit jobs.
> NullPointerException in fairshare PoolManager
> ---------------------------------------------
>
> Key: HADOOP-5389
> URL: https://issues.apache.org/jira/browse/HADOOP-5389
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.20.0
> Reporter: Nigel Daley
> Assignee: Vinod K V
>
> Found this NPE in a Hadoop 0.20.0 JobTracker log:
> {code}
> 2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5389) NullPointerException in fairshare
PoolManager
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala reassigned HADOOP-5389:
----------------------------------------
Assignee: Vinod K V
> NullPointerException in fairshare PoolManager
> ---------------------------------------------
>
> Key: HADOOP-5389
> URL: https://issues.apache.org/jira/browse/HADOOP-5389
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.20.0
> Reporter: Nigel Daley
> Assignee: Vinod K V
>
> Found this NPE in a Hadoop 0.20.0 JobTracker log:
> {code}
> 2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5389) NullPointerException in fairshare
PoolManager
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683881#action_12683881 ]
Hemanth Yamijala commented on HADOOP-5389:
------------------------------------------
IOW, this is the equivalent behavior of *-default.xml in hadoop. Maybe the message seems a little too alarming that something was wrong. If there was no file found, then we can simply say that there was no allocation file specified and will therefore use default allocation.
Would that work ?
> NullPointerException in fairshare PoolManager
> ---------------------------------------------
>
> Key: HADOOP-5389
> URL: https://issues.apache.org/jira/browse/HADOOP-5389
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.20.0
> Reporter: Nigel Daley
> Assignee: Vinod K V
>
> Found this NPE in a Hadoop 0.20.0 JobTracker log:
> {code}
> 2009-03-03 23:55:22,824 ERROR org.apache.hadoop.mapred.PoolManager: Failed to reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:222)
> at org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:119)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:226)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2340)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.