You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Devaraj K (JIRA)" <ji...@apache.org> on 2012/06/27 15:55:44 UTC
[jira] [Created] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Devaraj K created MAPREDUCE-4379:
------------------------------------
Summary: Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
Key: MAPREDUCE-4379
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
{code:xml}
Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedReader.<init>(BufferedReader.java:80)
at java.io.BufferedReader.<init>(BufferedReader.java:91)
at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:215)
at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406835#comment-13406835 ]
Devaraj K commented on MAPREDUCE-4379:
--------------------------------------
Can anyone have a look into this issue/patch?
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-4379:
---------------------------------
Status: Open (was: Patch Available)
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 2.0.0-alpha, 0.23.3, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403726#comment-13403726 ]
Hadoop QA commented on MAPREDUCE-4379:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533943/MAPREDUCE-4379.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
org.apache.hadoop.io.file.tfile.TestTFileByteArrays
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2531//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2531//console
This message is automatically generated.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403732#comment-13403732 ]
Devaraj K commented on MAPREDUCE-4379:
--------------------------------------
{code:xml}
-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-
project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
org.apache.hadoop.io.file.tfile.TestTFileByteArrays
{code}
These test failures are not related to this patch.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4379:
-------------------------------------------
Target Version/s: 0.23.3
Affects Version/s: 0.23.3
I really would like to see this go into 0.23 as well.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408164#comment-13408164 ]
Robert Joseph Evans commented on MAPREDUCE-4379:
------------------------------------------------
The change looks fine to me +1. I'll check this in. Please file the JIRA for cleaning up the caching.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-4379:
---------------------------------
Attachment: MAPREDUCE-4379-1.patch
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410825#comment-13410825 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2441 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.3, 2.0.1-alpha
>
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408295#comment-13408295 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2446 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2446/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408351#comment-13408351 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1128 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1128/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-4379:
---------------------------------
Status: Patch Available (was: Open)
Thanks a lot Robert for considering my request and looking into the patch.
I have updated the patch with above changes and ll file a separate jira to handle these things further.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 2.0.0-alpha, 0.23.3, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4379:
-------------------------------
Resolution: Fixed
Fix Version/s: 2.0.1-alpha
0.23.3
Target Version/s: (was: 0.23.3)
Status: Resolved (was: Patch Available)
This was committed to branch-2, branch-0.23 and trunk but was perhaps mistakenly not marked resolved. Updating Fix Versions and resolving.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.3, 2.0.1-alpha
>
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409859#comment-13409859 ]
Robert Joseph Evans commented on MAPREDUCE-4379:
------------------------------------------------
Thanks Harsh for resolving this. I meant to once I filed a follow up JIRA, but both you and Devaraj beat me to both of those.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.3, 2.0.1-alpha
>
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-4379:
---------------------------------
Attachment: MAPREDUCE-4379.patch
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409184#comment-13409184 ]
Devaraj K commented on MAPREDUCE-4379:
--------------------------------------
Thanks a lot Robert for reviewing and committing the patch.
I filed HADOOP-8578 for cleaning the cache.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.3, 2.0.1-alpha
>
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407976#comment-13407976 ]
Devaraj K commented on MAPREDUCE-4379:
--------------------------------------
{code:xml}-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
org.apache.hadoop.io.file.tfile.TestTFileByteArrays
{code}
These failing tests are not related to the patch.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402244#comment-13402244 ]
Devaraj K commented on MAPREDUCE-4379:
--------------------------------------
{code:title=ContainerLocalizer.java|borderStyle=solid}
this.appDirs =
new LocalDirAllocator(String.format(APPCACHE_CTXT_FMT, appId));
this.userDirs =
new LocalDirAllocator(String.format(USERCACHE_CTXT_FMT, appId));
this.pendingResources = new HashMap<LocalResource,Future<Path>>();
{code}
Here for every application during localization, it creates two LocalDirAllocator instances.
{code:title=LocalDirAllocator.java|borderStyle=solid}
private AllocatorPerContext obtainContext(String contextCfgItemName) {
synchronized (contexts) {
AllocatorPerContext l = contexts.get(contextCfgItemName);
if (l == null) {
contexts.put(contextCfgItemName,
(l = new AllocatorPerContext(contextCfgItemName)));
}
return l;
}
}
{code}
Those two instances will internally creates AllocatorPerContext instances and add those into contexts while obtaining contexts. It will keep on adding for every application and no where else these are getting removed from the map. It is leading to OOM after running for some time.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407968#comment-13407968 ]
Hadoop QA commented on MAPREDUCE-4379:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12535334/MAPREDUCE-4379-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
-1 javac. The applied patch generated 2070 javac compiler warnings (more than the trunk's current 2069 warnings).
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
org.apache.hadoop.io.file.tfile.TestTFileByteArrays
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2546//testReport/
Javac warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2546//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2546//console
This message is automatically generated.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408326#comment-13408326 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2428 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2428/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407189#comment-13407189 ]
Robert Joseph Evans commented on MAPREDUCE-4379:
------------------------------------------------
Sorry about that. I am looking at the patch now.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-4379:
---------------------------------
Priority: Blocker (was: Critical)
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408319#comment-13408319 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2496 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2496/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408288#comment-13408288 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1095 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1095/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410783#comment-13410783 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/])
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358305)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358305
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.3, 2.0.1-alpha
>
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4379.
------------------------------------
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.3, 2.0.2-alpha
>
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408369#comment-13408369 ]
Hudson commented on MAPREDUCE-4379:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #305 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/305/])
svn merge -c 1358305 FIXES: MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K via bobby) (Revision 1358312)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358312
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379-1.patch, MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407237#comment-13407237 ]
Robert Joseph Evans commented on MAPREDUCE-4379:
------------------------------------------------
Devaraj, could you upmerge the patch. It no longer applies to trunk.
For the most part the patch looks OK to me. It seems to fix the issue at hand. However, I have been reading through the code and I find a few things a bit disturbing. I don't really have a problem with your fix, but I think if we refactored the code it would make things a lot cleaner. Because this is a blocker I am inclined to just check it in, on condition that we mark LocalDirAllocator.removeContext as deprecated and limited private for MapReduce and file a separate JIRA to clean things up. Also this is only an issue if the DefaultContainerExecutor is enabled, because when the LinuxContainerExecutor is used the problematic code runs in a separate process that releases everything when the localization is done.
The way the code in LocalDirAllocator is written there are really only two reasons for the static cache. First to avoid doing extra file system calls when creating the LocalDirAllocator. The second is to keep track of the last dir written to for round robin access. In the case of the App Cache this is never going to save any FS calls, because there is 1 and only 1 LocalDirAllocator created per application, and the key to the cache is different for each application. For the User cache this would save a bit, except for a bug in the code that makes the LocalDirAllocator cache key for the user cache separate for all applications (and making the leak much worse).
{code} this.userDirs =
new LocalDirAllocator(String.format(USERCACHE_CTXT_FMT, appId));
...
conf.setStrings(String.format(USERCACHE_CTXT_FMT, appId), usersFileCacheDirs);
{code}
should be
{code} this.userDirs =
new LocalDirAllocator(String.format(USERCACHE_CTXT_FMT, user));
...
conf.setStrings(String.format(USERCACHE_CTXT_FMT, user), usersFileCacheDirs);
{code}
I would prefer to see the LocalDirAllocator cache have a maximum size, and have it remove entires in LRU order when it grows too big. This way we do not have to worry about removing entries we no longer need, and the worst thing that will happen is that if the cache is too small we will do some extra work, and potentially have disks written to in a more random order, instead of pure round robin.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4379) Node Manager throws
java.lang.OutOfMemoryError: Java heap space due to
org.apache.hadoop.fs.LocalDirAllocator.contexts
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-4379:
---------------------------------
Status: Patch Available (was: Open)
I have attached patch for this. Please review it.
> Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4379
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 2.0.0-alpha, 0.23.3, 3.0.0
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Blocker
> Attachments: MAPREDUCE-4379.patch
>
>
> {code:xml}
> Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
> at java.io.BufferedReader.<init>(BufferedReader.java:80)
> at java.io.BufferedReader.<init>(BufferedReader.java:91)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
> at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
> Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:215)
> at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
> at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
> at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
> at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira