You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/02/15 02:51:57 UTC
[jira] Created: (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
memory-related configurations missing from mapred-default.xml
-------------------------------------------------------------
Key: MAPREDUCE-2328
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Fix For: 0.22.0
HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2328:
-------------------------------------
Status: Open (was: Patch Available)
Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031631#comment-13031631 ]
Harsh J Chouraria commented on MAPREDUCE-2328:
----------------------------------------------
Justification: I believe this is mostly a doc-fix patch, and I've verified that no existing default-loading tests fail, as well as double checked the entered property names for typos/exact matches. The defaults were taken from the code's use of them around MR's code base.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054903#comment-13054903 ]
Hadoop QA commented on MAPREDUCE-2328:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12483814/MAPREDUCE-2328.r3.diff
against trunk revision 1139400.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.cli.TestMRCLI
org.apache.hadoop.fs.TestFileSystem
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/419//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/419//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/419//console
This message is automatically generated.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2328:
-------------------------------
Status: Patch Available (was: Open)
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria reassigned MAPREDUCE-2328:
--------------------------------------------
Assignee: Harsh J Chouraria
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138516#comment-13138516 ]
Todd Lipcon commented on MAPREDUCE-2328:
----------------------------------------
Only mapreduce.map.memory.mb and mapreduce.reduce.memory.mb are relevant in 23. The others are now yarn configs and have different names.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2328:
-------------------------------
Attachment: MAPREDUCE-2328.r3.diff
Improved explanations for the cluster wide properties.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138227#comment-13138227 ]
Harsh J commented on MAPREDUCE-2328:
------------------------------------
Hello,
Heading to rebase this. Would all of these properties still be valid on 0.23?
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2328:
-------------------------------
Attachment: MAPREDUCE-2328.r2.diff
New patch with additions and docs improvement based on Todd's comments and links (and mechanisms of how the cluster-wide properties are utilized.)
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2328:
-----------------------------------------
Status: Open (was: Patch Available)
Thanks Todd. I'll put up a fixed patch shortly.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2328:
-----------------------------------------
Attachment: MAPREDUCE-2328.r1.diff
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135502#comment-13135502 ]
Konstantin Shvachko commented on MAPREDUCE-2328:
------------------------------------------------
Sorry to come even later.
This does apply to 0.22 and would be good to have it in the release.
But a patch for trunk is needed.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047960#comment-13047960 ]
Hadoop QA commented on MAPREDUCE-2328:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12482154/MAPREDUCE-2328.r2.diff
against trunk revision 1134180.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.cli.TestMRCLI
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.examples.terasort.TestTeraSort
org.apache.hadoop.fs.TestDFSIO
org.apache.hadoop.fs.TestFileSystem
org.apache.hadoop.ipc.TestSocketFactory
org.apache.hadoop.mapred.join.TestDatamerge
org.apache.hadoop.mapred.lib.TestDelegatingInputFormat
org.apache.hadoop.mapred.pipes.TestPipes
org.apache.hadoop.mapred.TestBadRecords
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.mapred.TestCommandLineJobSubmission
org.apache.hadoop.mapred.TestCompressedEmptyMapOutputs
org.apache.hadoop.mapred.TestControlledMapReduceJob
org.apache.hadoop.mapred.TestFileInputFormat
org.apache.hadoop.mapred.TestJobClient
org.apache.hadoop.mapred.TestJobDirCleanup
org.apache.hadoop.mapred.TestJobHistory
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapred.TestJobQueueInformation
org.apache.hadoop.mapred.TestJobStatusPersistency
org.apache.hadoop.mapred.TestJobSysDirWithDFS
org.apache.hadoop.mapred.TestJobTrackerXmlJsp
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapred.TestMapredHeartbeat
org.apache.hadoop.mapred.TestMapredSystemDir
org.apache.hadoop.mapred.TestMiniMRChildTask
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestMiniMRDFSCaching
org.apache.hadoop.mapred.TestMiniMRDFSSort
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapred.TestMiniMRWithDFS
org.apache.hadoop.mapred.TestMultipleLevelCaching
org.apache.hadoop.mapred.TestNodeRefresh
org.apache.hadoop.mapred.TestRecoveryManager
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapred.TestSetupAndCleanupFailure
org.apache.hadoop.mapred.TestSeveral
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapred.TestSubmitJob
org.apache.hadoop.mapred.TestTaskFail
org.apache.hadoop.mapred.TestWebUIAuthorization
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
org.apache.hadoop.mapreduce.lib.input.TestDelegatingInputFormat
org.apache.hadoop.mapreduce.lib.join.TestJoinDatamerge
org.apache.hadoop.mapreduce.lib.join.TestJoinProperties
org.apache.hadoop.mapreduce.security.TestBinaryTokenFile
org.apache.hadoop.mapreduce.security.TestTokenCacheOldApi
org.apache.hadoop.mapreduce.security.TestTokenCache
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapreduce.TestMRJobClient
org.apache.hadoop.security.authorize.TestServiceLevelAuthorization
org.apache.hadoop.security.TestMapredGroupMappingServiceRefresh
org.apache.hadoop.tools.TestCopyFiles
org.apache.hadoop.tools.TestDistCh
org.apache.hadoop.tools.TestHadoopArchives
org.apache.hadoop.tools.TestHarFileSystem
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/377//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/377//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/377//console
This message is automatically generated.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138401#comment-13138401 ]
Hudson commented on MAPREDUCE-2328:
-----------------------------------
Integrated in Hadoop-Mapreduce-22-branch #85 (See [https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/85/])
MAPREDUCE-2328. Add memory-related configuration parameters to mapred-default.xml. Contributed by Harsh J.
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190173
Files :
* /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt
* /hadoop/common/branches/branch-0.22/mapreduce/src/java/mapred-default.xml
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2328:
-------------------------------
Target Version/s: 2.0.1-alpha
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.1
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032293#comment-13032293 ]
Todd Lipcon commented on MAPREDUCE-2328:
----------------------------------------
Seems we are still missing mapreduce.map.memory.mb, mapreduce.map.memory.physical.mb, etc from mapred-default.xml.
I think the descriptions could also be improved. Take a look at src/docs/src/documentation/content/xdocs/cluster_setup.xml in the common repo for pretty good descriptions, as well as [this comment|https://issues.apache.org/jira/browse/HADOOP-5881?focusedCommentId=12712751&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12712751].
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2328:
-----------------------------------------
Status: Patch Available (was: Open)
Patch with brief description of the options, in mapred-default.xml.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138108#comment-13138108 ]
Konstantin Shvachko commented on MAPREDUCE-2328:
------------------------------------------------
I committed this to 0.22 branch. Thank you Harsh.
Leaving it open for the action on trunk / 0.23
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, MAPREDUCE-2328.r3.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations
missing from mapred-default.xml
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031621#comment-13031621 ]
Hadoop QA commented on MAPREDUCE-2328:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12478782/MAPREDUCE-2328.r1.diff
against trunk revision 1101741.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/228//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/228//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/228//console
This message is automatically generated.
> memory-related configurations missing from mapred-default.xml
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2328
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Harsh J Chouraria
> Labels: newbie
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2328.r1.diff
>
>
> HADOOP-5881 added new configuration parameters for memory-based scheduling, but they weren't added to mapred-default.xml
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira