You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ahmed Radwan (Created) (JIRA)" <ji...@apache.org> on 2012/02/28 05:58:48 UTC
[jira] [Created] (MAPREDUCE-3933) Failures because MALLOC_ARENA_MAX
is not set
Failures because MALLOC_ARENA_MAX is not set
--------------------------------------------
Key: MAPREDUCE-3933
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, test
Affects Versions: 0.23.0
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
These tests fail with messages of the form:
{code}
[Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
{code}
The failing tests are:
{code}
TestJobCounters
TestJobSysDirWithDFS
TestLazyOutput
TestMiniMRChildTask
TestMiniMRClientCluster
TestReduceFetchFromPartialMem
TestChild
TestMapReduceLazyOutput
TestJobOutputCommitter
TestMRAppWithCombiner
TestMRJobs
TestMRJobsWithHistoryService
TestMROldApiJobs
TestSpeculativeExecution
TestUberAM
{code}
I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3933) Failures because MALLOC_ARENA_MAX
is not set
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3933:
------------------------------------
Attachment: MAPREDUCE-3933.patch
A patch setting MALLOC_ARENA_MAX in the root pom.xml. I also removed this setting from DistributedShell pom as it is no longer needed when it is added to the root pom. I have verified that all failing tests now run successfully.
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-3933.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219332#comment-13219332 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1807 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1807/])
MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu) (Revision 1295178)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295178
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219331#comment-13219331 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #617 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/617/])
Merge -r 1295177:1295178 from trunk to branch. FIXES: MAPREDUCE-3933 (Revision 1295179)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295179
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220002#comment-13220002 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #971 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/971/])
MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu) (Revision 1295178)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295178
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3933) Failures because MALLOC_ARENA_MAX
is not set
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3933:
------------------------------------
Attachment: MAPREDUCE-3933_rev2.patch
Updated patch that modifies the pom under hadoop-project instead of the root pom. Changes to any is resolving the issue but based on an offline discussion with Tucu, he advised to use the pom in hadoop-project, similar to the root pom, any variables set there are inherited by all modules. Thanks Tucu!
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217911#comment-13217911 ]
Hadoop QA commented on MAPREDUCE-3933:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516276/MAPREDUCE-3933.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1945//console
This message is automatically generated.
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-3933.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3933) Failures because MALLOC_ARENA_MAX
is not set
Posted by "Alejandro Abdelnur (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-3933:
------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.3
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Ahmed. Committed to trunk and branch-0.23
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219327#comment-13219327 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1881 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1881/])
MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu) (Revision 1295178)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295178
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220050#comment-13220050 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1006 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1006/])
MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu) (Revision 1295178)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295178
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219353#comment-13219353 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #617 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/617/])
Merge -r 1295177:1295178 from trunk to branch. FIXES: MAPREDUCE-3933 (Revision 1295179)
Result = ABORTED
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295179
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218839#comment-13218839 ]
Hadoop QA commented on MAPREDUCE-3933:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516495/MAPREDUCE-3933_rev2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1958//console
This message is automatically generated.
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219346#comment-13219346 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1815 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1815/])
MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu) (Revision 1295178)
Result = ABORTED
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295178
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219328#comment-13219328 ]
Hudson commented on MAPREDUCE-3933:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #605 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/605/])
Merge -r 1295177:1295178 from trunk to branch. FIXES: MAPREDUCE-3933 (Revision 1295179)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295179
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3933) Failures because MALLOC_ARENA_MAX
is not set
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3933:
------------------------------------
Status: Patch Available (was: Open)
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-3933.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3933) Failures because
MALLOC_ARENA_MAX is not set
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218920#comment-13218920 ]
Alejandro Abdelnur commented on MAPREDUCE-3933:
-----------------------------------------------
+1
> Failures because MALLOC_ARENA_MAX is not set
> --------------------------------------------
>
> Key: MAPREDUCE-3933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3933
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: MAPREDUCE-3933.patch, MAPREDUCE-3933_rev2.patch
>
>
> We have noticed a bunch of MapReduce test failures on CentOS 6 due to "running beyond virtual memory limits".
> These tests fail with messages of the form:
> {code}
> [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNodeStatus(254)) - Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1330401645767, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: "Container [pid=16750,containerID=container_1330401645767_0001_01_000001] is running beyond virtual memory limits. Current usage: 220.5mb of 2.0gb physical memory used; 7.1gb of 4.2gb virtual memory used. Killing container
> {code}
> The failing tests are:
> {code}
> TestJobCounters
> TestJobSysDirWithDFS
> TestLazyOutput
> TestMiniMRChildTask
> TestMiniMRClientCluster
> TestReduceFetchFromPartialMem
> TestChild
> TestMapReduceLazyOutput
> TestJobOutputCommitter
> TestMRAppWithCombiner
> TestMRJobs
> TestMRJobsWithHistoryService
> TestMROldApiJobs
> TestSpeculativeExecution
> TestUberAM
> {code}
> I'll upload a patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira