You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (JIRA)" <ji...@apache.org> on 2012/04/22 15:31:37 UTC
[jira] [Created] (MAPREDUCE-4177) Clean up
hadoop-mapreduce-client-jobclient
Harsh J created MAPREDUCE-4177:
----------------------------------
Summary: Clean up hadoop-mapreduce-client-jobclient
Key: MAPREDUCE-4177
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4177
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: build
Affects Versions: trunk
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Clean up a bunch of existing javac warnings in hadoop-mapreduce-client-jobclient module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4177) Clean up
hadoop-mapreduce-client-jobclient
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4177:
-------------------------------
Status: Patch Available (was: Open)
> Clean up hadoop-mapreduce-client-jobclient
> ------------------------------------------
>
> Key: MAPREDUCE-4177
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4177
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: build
> Affects Versions: trunk
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Attachments: 0005-MapReduce-Client-JobClient-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-mapreduce-client-jobclient module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4177) Clean up
hadoop-mapreduce-client-jobclient
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4177:
-------------------------------
Attachment: 0005-MapReduce-Client-JobClient-Cleanup.patch
Attached patch cleans up almost all of the code and tests save for some MiniMRCluster and DistributedCache references for which its best to revisit and open a new JIRA (will do upon parent completion).
> Clean up hadoop-mapreduce-client-jobclient
> ------------------------------------------
>
> Key: MAPREDUCE-4177
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4177
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: build
> Affects Versions: trunk
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Attachments: 0005-MapReduce-Client-JobClient-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-mapreduce-client-jobclient module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4177) Clean up
hadoop-mapreduce-client-jobclient
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259188#comment-13259188 ]
Hadoop QA commented on MAPREDUCE-4177:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523692/0005-MapReduce-Client-JobClient-Cleanup.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 130 new or modified test files.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.TestContainerManagerSecurity
org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
org.apache.hadoop.mapred.TestMapRed
org.apache.hadoop.mapred.TestClientRedirect
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
org.apache.hadoop.mapreduce.security.TestJHSSecurity
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2274//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2274//console
This message is automatically generated.
> Clean up hadoop-mapreduce-client-jobclient
> ------------------------------------------
>
> Key: MAPREDUCE-4177
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4177
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: build
> Affects Versions: trunk
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Attachments: 0005-MapReduce-Client-JobClient-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-mapreduce-client-jobclient module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4177) Clean up
hadoop-mapreduce-client-jobclient
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4177:
-------------------------------
Attachment: 0005-MapReduce-Client-JobClient-Cleanup.patch
The relevant test failures here are:
{{org.apache.hadoop.mapred.TestMapRed}}
This failed cause I fixed the reduce() signature in MyReducer (which extended IdentityReducer). Doing so activated it to run in real. All these months (years?) its been running just the identity signature cause the provided implementation had no @Override. The method in reduce() also was old - from a time back when SequenceFile was used as the intermediate format (it tests if the sequence file is compressed if map-output compression is on). I got rid of this check as there are checks elsewhere that guarantee this now. The reducer now used is purely identity and hence the test shall no longer fail.
{{org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat}}
This was cause one of the DummyRecordReader constructors was removed (Cause it was marked as unused by javac). I added it back and test passes again, so I threw in a comment on why we should keep it and suppressed the unused warning.
Other test failures are irrelevant to the patch and have been failing intermittently across all MAPREDUCE-4172 (and others) QA builds.
Newly attached patch should fix all the real issues that arose here (as described above).
> Clean up hadoop-mapreduce-client-jobclient
> ------------------------------------------
>
> Key: MAPREDUCE-4177
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4177
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: build
> Affects Versions: trunk
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Attachments: 0005-MapReduce-Client-JobClient-Cleanup.patch, 0005-MapReduce-Client-JobClient-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-mapreduce-client-jobclient module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4177) Clean up
hadoop-mapreduce-client-jobclient
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4177:
-------------------------------
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
Patches were too broad and have gone stale. Will address these forms of issue over separate, smaller and more divided JIRAs in future.
Closing out parent JIRA MAPREDUCE-4172, and hence closing out this.
> Clean up hadoop-mapreduce-client-jobclient
> ------------------------------------------
>
> Key: MAPREDUCE-4177
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4177
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: build
> Affects Versions: trunk
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Attachments: 0005-MapReduce-Client-JobClient-Cleanup.patch, 0005-MapReduce-Client-JobClient-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-mapreduce-client-jobclient module.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira