You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinay Kumar Thota (JIRA)" <ji...@apache.org> on 2011/06/01 18:42:47 UTC
[jira] [Created] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Gridmix distributed cache emulation system tests.
-------------------------------------------------
Key: MAPREDUCE-2554
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
Project: Hadoop Map/Reduce
Issue Type: Task
Components: contrib/gridmix
Reporter: Vinay Kumar Thota
Assignee: Vinay Kumar Thota
1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
4.Verify the emulation of local FS distributed cache files.
5.Verify the Gridmix emulation of HDFS private distributed cache file.
6.Verify the Gridmix emulation of HDFS public distributed cache file.
7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057258#comment-13057258 ]
Hudson commented on MAPREDUCE-2554:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #722 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/])
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated MAPREDUCE-2554:
----------------------------------
Resolution: Fixed
Release Note: Adds distributed cache related system tests to Gridmix.
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed the latest patch to trunk on Ravi's behalf. Thanks Vinay and Ravi!
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-2554:
-----------------------------------------
Attachment: MAPREDUCE-2554.patch
Please review the patch and let me know the comments.
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043384#comment-13043384 ]
Hadoop QA commented on MAPREDUCE-2554:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12481316/MAPREDUCE-2554-v2.patch
against trunk revision 1130994.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 46 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.cli.TestMRCLI
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/341//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/341//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/341//console
This message is automatically generated.
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043319#comment-13043319 ]
Ravi Gummadi commented on MAPREDUCE-2554:
-----------------------------------------
Patch looks fine to me.
+1
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-2554:
-----------------------------------------
Fix Version/s: 0.23.0
Status: Patch Available (was: Open)
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-2554:
-----------------------------------------
Attachment: MAPREDUCE-2554-v2.patch
Uploaded another new patch with minor changes.Ran the system tests with the patch and no issues were found.
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044428#comment-13044428 ]
Hudson commented on MAPREDUCE-2554:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #713 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/713/])
MAPREDUCE-2554. [Gridmix] Add distributed cache emulation system tests to Gridmix. (Vinay Kumar Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131515
Files :
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case7_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case8_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case9_trace.json.gz
* /hadoop/mapreduce/trunk/CHANGES.txt
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/DistributedCacheEmulator.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestGridmixEmulationOfMultipleHDFSPrivateDCFiles.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestGridmixEmulationOfHDFSPublicDCFile.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestEmulationOfLocalFSDCFiles.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/UtilsForGridmix.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestEmulationOfHDFSAndLocalFSDCFiles.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestEmulationOfHDFSDCFileUsesMultipleJobs.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestEmulationOfHDFSDCFilesWithDifferentVisibilities.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestGridmixEmulationOfMultipleHDFSPublicDCFiles.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestGridmixEmulationOfHDFSPrivateDCFile.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case1_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case2_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case3_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/GridmixSystemTestCase.java
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case4_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case5_trace.json.gz
* /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/system/resources/distcache_case6_trace.json.gz
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2554) Gridmix distributed cache
emulation system tests.
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044420#comment-13044420 ]
Vinay Kumar Thota commented on MAPREDUCE-2554:
----------------------------------------------
Findbugs are not related to this patch.Also TestMRCLI failure is not related this patch.
> Gridmix distributed cache emulation system tests.
> -------------------------------------------------
>
> Key: MAPREDUCE-2554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2554
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2554-v2.patch, MAPREDUCE-2554.patch
>
>
> 1.Verify the emulation of HDFS and Local FS distributed cache files against the given input trace file.
> 2.Verify the Gridmix emulation of HDFS distributed cache files of different visibilities.
> 3.Verify the Gridmix emulation of HDFS distributed cache file which uses different jobs that are submitted with different users.
> 4.Verify the emulation of local FS distributed cache files.
> 5.Verify the Gridmix emulation of HDFS private distributed cache file.
> 6.Verify the Gridmix emulation of HDFS public distributed cache file.
> 7.Verify the Gridmix emulation of Multiple HDFS private distributed cache files.
> 8.Verify the Gridmix emulation of Multiple HDFS public distributed cache files.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira