You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2012/08/30 14:30:07 UTC
[jira] [Created] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Tom White created MAPREDUCE-4610:
------------------------------------
Summary: Support deprecated mapreduce.job.counters.limit property in MR2
Key: MAPREDUCE-4610
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0-alpha
Reporter: Tom White
Assignee: Tom White
The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445890#comment-13445890 ]
Hudson commented on MAPREDUCE-4610:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1151 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1151/])
MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in MR2. (Revision 1379022)
Result = FAILURE
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379022
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464669#comment-13464669 ]
Hudson commented on MAPREDUCE-4610:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #387 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/387/])
svn merge -c 1379022 FIXES: MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in MR2. (Revision 1390670)
Result = UNSTABLE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390670
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 2.0.2-alpha
>
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445110#comment-13445110 ]
Philip Zeyliger commented on MAPREDUCE-4610:
--------------------------------------------
+1 LGTM.
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated MAPREDUCE-4610:
---------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this.
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445945#comment-13445945 ]
Hudson commented on MAPREDUCE-4610:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1182/])
MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in MR2. (Revision 1379022)
Result = SUCCESS
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379022
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4610.
------------------------------------
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 2.0.2-alpha
>
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated MAPREDUCE-4610:
---------------------------------
Attachment: MAPREDUCE-4610.patch
Here's a patch that fixes this. I tested it manually by starting a cluster and running a job that creates more than the default number of counters (120). When I increased mapreduce.job.counters.limit beyond that number I didn't see any counter limit exception.
Writing a unit test is actually difficult since the limits are loaded statically. I'd like to fix that as a part of fixing the client-side limit enforcement problem in MAPREDUCE-3936. But I think this is useful to go in as a compatibility fix first.
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445043#comment-13445043 ]
Alejandro Abdelnur commented on MAPREDUCE-4610:
-----------------------------------------------
+1. LGTM.
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated MAPREDUCE-4610:
---------------------------------
Status: Patch Available (was: Open)
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444982#comment-13444982 ]
Hadoop QA commented on MAPREDUCE-4610:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12543096/MAPREDUCE-4610.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javac. The applied patch generated 2059 javac compiler warnings (more than the trunk's current 2058 warnings).
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2791//testReport/
Javac warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2791//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2791//console
This message is automatically generated.
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4610) Support deprecated
mapreduce.job.counters.limit property in MR2
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444989#comment-13444989 ]
Tom White commented on MAPREDUCE-4610:
--------------------------------------
The extra javac warning is from the Configuration.addDeprecation which follows the style of the other ones in ConfigUtil, so is harmless.
> Support deprecated mapreduce.job.counters.limit property in MR2
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4610
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-4610.patch
>
>
> The property mapreduce.job.counters.limit was introduced in MAPREDUCE-1943, but the mechanism was changed in MAPREDUCE-901 where the property name was changed to mapreduce.job.counters.max without supporting the old name. We should deprecate but honour the old name to make it easier for folks to move from Hadoop 1 to Hadoop 2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira