You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinay Kumar Thota (Created) (JIRA)" <ji...@apache.org> on 2011/11/08 09:18:51 UTC
[jira] [Created] (MAPREDUCE-3375) Memory Emulation system tests.
Memory Emulation system tests.
------------------------------
Key: MAPREDUCE-3375
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
Project: Hadoop Map/Reduce
Issue Type: Task
Reporter: Vinay Kumar Thota
Assignee: Vinay Kumar Thota
1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150478#comment-13150478 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #898 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/898/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201997
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150465#comment-13150465 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #94 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/94/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201999
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150185#comment-13150185 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #170 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/170/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201999
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149733#comment-13149733 ]
Vinay Kumar Thota commented on MAPREDUCE-3375:
----------------------------------------------
Ran test-patch manually.
+1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 20 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Amar Kamat (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat resolved MAPREDUCE-3375.
-----------------------------------
Tags: gridmix system-tests memory-emulation
Resolution: Fixed
Fix Version/s: 0.23.1
0.24.0
Target Version/s: 0.24.0
Release Note: Added system tests to test the memory emulation feature in Gridmix.
Hadoop Flags: Reviewed
I just committed this to trunk and branch-0.23. Thanks Vinay!
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150191#comment-13150191 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/182/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201999
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150175#comment-13150175 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1347 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1347/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201997
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-3375:
-----------------------------------------
Attachment: MAPREDUCE-3375.v3.patch
Added one more scenario for total heap ratio.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-3375:
-----------------------------------------
Attachment: MAPREDUCE-3375.v1.patch
Memory emulation system tests initial patch. Please review it and let me know your comments.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-3375:
-----------------------------------------
Attachment: MAPREDUCE-3375.v5.patch
Amar, incorporated your comments.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149514#comment-13149514 ]
Vinay Kumar Thota commented on MAPREDUCE-3375:
----------------------------------------------
Ran tests with latest patch and all the tests are passed.
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsAndReducesWithCustomIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 400.59 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsAndReducesWithDefaultIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 440.233 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsWithCustomHeapMemoryRatio.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 650.089 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsWithCustomIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 674.838 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsWithDefaultIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 605.841 sec
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150178#comment-13150178 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1296 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1296/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201997
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-3375:
-----------------------------------------
Attachment: MAPREDUCE-3375.v2.patch
Addressed Amar's offline comments.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149590#comment-13149590 ]
Vinay Kumar Thota commented on MAPREDUCE-3375:
----------------------------------------------
Ran test-patch and no issues found.
+1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-3375:
-----------------------------------------
Attachment: MAPREDUCE-3375.v4.patch
Uploading another new patch by addressing some more offline comments from Amar.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150179#comment-13150179 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1273 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1273/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201997
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150446#comment-13150446 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #77 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/77/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201999
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148398#comment-13148398 ]
Vinay Kumar Thota commented on MAPREDUCE-3375:
----------------------------------------------
Ran system tests and all the tests are passed.
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsAndReducesWithCustomIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 411.758 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsAndReducesWithDefaultIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 364.966 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsWithCustomIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 626.893 sec
TEST-org.apache.hadoop.mapred.gridmix.TestMemEmulForMapsWithDefaultIntrvl.txt:Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 705.531 sec
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150189#comment-13150189 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #171 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/171/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201999
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150437#comment-13150437 ]
Hudson commented on MAPREDUCE-3375:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #864 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/864/])
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests. (Vinay Thota via amarrk)
amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201997
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsAndReducesWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomHeapMemoryRatio.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithCustomIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/TestMemEmulForMapsWithDefaultIntrvl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridMixConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/org/apache/hadoop/mapred/gridmix/test/system/GridmixJobVerification.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case1.json.gz
* /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/system/resources/mem_emul_case2.json.gz
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.24.0, 0.23.1
>
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch, MAPREDUCE-3375.v5.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Vinay Kumar Thota (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148418#comment-13148418 ]
Vinay Kumar Thota commented on MAPREDUCE-3375:
----------------------------------------------
Ran test-patch and no issues found.
+1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3375) Memory Emulation system tests.
Posted by "Amar Kamat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149602#comment-13149602 ]
Amar Kamat commented on MAPREDUCE-3375:
---------------------------------------
Vinay,
The assert statement
{noformat}
Assert.assertTrue("Simulated job's map-task heap ratio did not match.", actHeapRatio <= expHeapRatio );
{noformat}
needs a fix. In most of the cases, {{actHeapRatio}} will be 0 as its a division between 2 longs. So firstly the statement
{noformat}
double actHeapRatio = simuJobMapsTHU / origJobMapsTHU;
{noformat}
should be
{noformat}
double actHeapRatio = ((double)simuJobMapsTHU) / origJobMapsTHU;
{noformat}
or something like that.
Secondly, the gap between the original and simulated tasks' heap memory usage should be checked and not the actual usages. Hence, the above mentioned formula now becomes
{noformat}
double actHeapRatio = ((double)Math.abs(origJobMapsTHU - simuJobMapsTHU)) / origJobMapsTHU;
{noformat}
Kindly make the modifications and re-run the system test.
> Memory Emulation system tests.
> ------------------------------
>
> Key: MAPREDUCE-3375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3375
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: MAPREDUCE-3375.v1.patch, MAPREDUCE-3375.v2.patch, MAPREDUCE-3375.v3.patch, MAPREDUCE-3375.v4.patch
>
>
> 1. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 2. Test the Gridmix memory emulation feature for gridmix jobs with custom progress interval, different input data, submission policies and user resolver modes . Verify the maps phase of total heap usage of gridmix jobs with corresponding the original job in the trace.
> 3. Test the Gridmix memory emulation feature for gridmix jobs with default progress interval, different input data, submission policies and user resolver modes. Verify the maps and reduces phase of total heap usage metric of gridmix jobs with corresponding the original job in the trace.
> 4. Disable Gridmix memory emulation option and verify the jobs whether it emulates the heap memory or not.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira