You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Karam Singh (Created) (JIRA)" <ji...@apache.org> on 2011/12/12 06:38:30 UTC
[jira] [Created] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws NPE causes scheduling to stop
Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop
---------------------------------------------------------------------------
Key: MAPREDUCE-3530
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager, scheduler
Reporter: Karam Singh
Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
I have been observing intermitently for last 3 weeks.
But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
{code}
java.lang.NullPointerException
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
at java.lang.Thread.run(Thread.java:619)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws NPE causes scheduling to stop
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3530:
-------------------------------------
Priority: Blocker (was: Major)
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Reporter: Karam Singh
> Priority: Blocker
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3530:
-------------------------------------
Attachment: MAPREDUCE-3530.patch
Same patch, alongwith a unit test.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168950#comment-13168950 ]
Siddharth Seth commented on MAPREDUCE-3530:
-------------------------------------------
The RM should probably exit if the scheduler thread sees exceptions, instead of the RM continuing to run without the scheduler thread. That could be changed here or in a separate jira ?
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws NPE causes scheduling to stop
Posted by "Arun C Murthy (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned MAPREDUCE-3530:
----------------------------------------
Assignee: Vinod Kumar Vavilapalli
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169004#comment-13169004 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3530:
----------------------------------------------------
bq. The RM should probably exit if the scheduler thread sees exceptions, instead of the RM continuing to run without the scheduler thread.
Let's do that separately. We need this kind of checking for all components.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3530:
-----------------------------------------------
Resolution: Fixed
Release Note: Fixed an NPE occuring during scheduling in the ResourceManager.
Status: Resolved (was: Patch Available)
Just committed this to trunk and branch-0.23. Thanks Arun!
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169239#comment-13169239 ]
Hadoop QA commented on MAPREDUCE-3530:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507322/MAPREDUCE-3530.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1444//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1444//console
This message is automatically generated.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3530:
-----------------------------------------------
Fix Version/s: 0.23.1
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3530:
-------------------------------------
Target Version/s: 0.23.1
Status: Patch Available (was: Open)
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169780#comment-13169780 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #279 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/279/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
svn merge -c 1214476 --ignore-ancestry ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214478
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3530:
-------------------------------------
Status: Open (was: Patch Available)
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws NPE causes scheduling to stop
Posted by "Arun C Murthy (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned MAPREDUCE-3530:
----------------------------------------
Assignee: Arun C Murthy (was: Vinod Kumar Vavilapalli)
I'll take a look.
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws NPE causes scheduling to stop
Posted by "Karam Singh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karam Singh updated MAPREDUCE-3530:
-----------------------------------
Affects Version/s: 0.23.1
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Priority: Blocker
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3530:
-----------------------------------------------
Attachment: MAPREDUCE-3530-20111214.txt
Updated patch with the trivial edits.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168968#comment-13168968 ]
Hadoop QA commented on MAPREDUCE-3530:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507272/MAPREDUCE-3530.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1440//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1440//console
This message is automatically generated.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3530:
-------------------------------------
Attachment: MAPREDUCE-3530.patch
With Vinod's help, fixed a corner case where expired/released containers aren't ignored when NM reports back (possibly late).
I'm adding a unit test.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169732#comment-13169732 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #301 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/301/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
svn merge -c 1214476 --ignore-ancestry ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214478
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws NPE causes scheduling to stop
Posted by "Karam Singh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167363#comment-13167363 ]
Karam Singh commented on MAPREDUCE-3530:
----------------------------------------
Vinod was aware of it and was working on it
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Reporter: Karam Singh
> Priority: Blocker
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169922#comment-13169922 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #126 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/126/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
svn merge -c 1214476 --ignore-ancestry ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214478
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170226#comment-13170226 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #108 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/108/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
svn merge -c 1214476 --ignore-ancestry ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214478
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170292#comment-13170292 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #928 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/928/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214476
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169775#comment-13169775 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1506 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1506/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214476
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169772#comment-13169772 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #289 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/289/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
svn merge -c 1214476 --ignore-ancestry ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214478
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169777#comment-13169777 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1433 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1433/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214476
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3530:
-----------------------------------------------
Summary: Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop (was: Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop)
While I was testing with Karam, I worked around it by putting in a null check. We have to come up with a scenario, I couldn't find much from the logs too at that time.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169734#comment-13169734 ]
Hudson commented on MAPREDUCE-3530:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1456 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1456/])
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the ResourceManager. Contributed by Arun C Murthy.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214476
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3530:
-----------------------------------------------
Status: Open (was: Patch Available)
Patch looks good overall.
Minor edits:
- RNNodeImpl.getStateMachine() isn't used anywhere.
- The real ContainerID etc are as easy to be created using BuilderUtils.
I'll upload an updated patch with these trivial edits and push this in after Jenkins blesses it.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3530:
-------------------------------------
Status: Patch Available (was: Open)
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3530) Sometimes NODE_UPDATE to the
scheduler throws an NPE causing the scheduling to stop
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169657#comment-13169657 ]
Hadoop QA commented on MAPREDUCE-3530:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507398/MAPREDUCE-3530-20111214.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1451//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1451//console
This message is automatically generated.
> Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
> -----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3530
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3530
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3530-20111214.txt, MAPREDUCE-3530.patch, MAPREDUCE-3530.patch
>
>
> Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
> I have been observing intermitently for last 3 weeks.
> But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira