You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/04/22 04:35:05 UTC
[jira] [Created] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
TestMiniMRWithDFSWithDistinctUsers is very broken
-------------------------------------------------
Key: MAPREDUCE-2445
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: security, test
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Fix For: 0.22.0
This test has a number of issues:
- it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
- it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
- it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
- it shuts down DFS before MR
These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026071#comment-13026071 ]
Hudson commented on MAPREDUCE-2445:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #647 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/647/])
MAPREDUCE-2445. Fix TestMiniMRWithDFSWithDistinctUsers to be a valid test. Contributed by Todd Lipcon.
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023091#comment-13023091 ]
Hadoop QA commented on MAPREDUCE-2445:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12477071/mapreduce-2445.txt
against trunk revision 1094093.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The patch appears to cause tar ant target to fail.
-1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
-1 contrib tests. The patch failed contrib unit tests.
-1 system test framework. The patch failed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/177//testReport/
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/177//console
This message is automatically generated.
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2445:
-----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk and 22, thanks Tom.
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028211#comment-13028211 ]
Hudson commented on MAPREDUCE-2445:
-----------------------------------
Integrated in Hadoop-Mapreduce-22-branch #44 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/44/])
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023132#comment-13023132 ]
Hadoop QA commented on MAPREDUCE-2445:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12477074/mapreduce-2445.txt
against trunk revision 1094093.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/178//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/178//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/178//console
This message is automatically generated.
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2445:
-----------------------------------
Status: Patch Available (was: Open)
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028779#comment-13028779 ]
Hudson commented on MAPREDUCE-2445:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #669 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/669/])
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025955#comment-13025955 ]
Tom White commented on MAPREDUCE-2445:
--------------------------------------
+1
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2445:
-----------------------------------
Attachment: mapreduce-2445.txt
Oops, uploaded an incomplete patch. Take 2.
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt, mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2445)
TestMiniMRWithDFSWithDistinctUsers is very broken
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-2445:
-----------------------------------
Attachment: mapreduce-2445.txt
Here's a fixed up test case:
- renamed "wc" and "pi" users to "alice" and "bob", since in fact both are running wordcount
- set up a proper staging directory with sticky bit, etc, so job can be successfully submitted
- sets up home directories for each user that are only writable by them (to make sure that the job is running as the right users)
- avoids using internal job submission APIs, so job actually submits
- checks return status of job to actually assert success
- shuts down services in correct order
> TestMiniMRWithDFSWithDistinctUsers is very broken
> -------------------------------------------------
>
> Key: MAPREDUCE-2445
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2445
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, test
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-2445.txt
>
>
> This test has a number of issues:
> - it side steps the normal job submission API for no apparent reason, manually writing splits file and uploading submission files. (but forgets to upload the job jar, so the jobs all fail)
> - it doesn't call waitForCompletion, or check job status (so it doesn't notice that the jobs all fail)
> - it doesn't verify in any way that the job output is owned by the user who supposedly ran the job
> - it shuts down DFS before MR
> These all conspire to make it pass, but it isn't actually testing anything.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira