You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2009/03/09 06:40:50 UTC
[jira] Created: (HADOOP-5437) Unit test for jvm -reuse
Unit test for jvm -reuse
------------------------
Key: HADOOP-5437
URL: https://issues.apache.org/jira/browse/HADOOP-5437
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.19.0
Reporter: Amareshwari Sriramadasu
Priority: Minor
Fix For: 0.21.0
TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695662#action_12695662 ]
Chris Douglas commented on HADOOP-5437:
---------------------------------------
+1
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Priority: Blocker (was: Minor)
Fix Version/s: (was: 0.21.0)
0.20.0
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695981#action_12695981 ]
Hadoop QA commented on HADOOP-5437:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12404690/h5437.patch
against trunk revision 761838.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/116/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/116/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/116/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/116/console
This message is automatically generated.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680739#action_12680739 ]
Amareshwari Sriramadasu commented on HADOOP-5437:
-------------------------------------------------
Test failure TestJobHistory is not related to the patch.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Attachment: h5437.patch
This patch:
1. Sets the mappers to 12 for sort.
2. Uses the test setup and splits the test cases.
3. Increases the data per a map to 3mb, to guarantee a spill.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695654#action_12695654 ]
Hadoop QA commented on HADOOP-5437:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12404611/h5437.patch
against trunk revision 761838.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/114/console
This message is automatically generated.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Attachment: h5437.patch
The complete patch for the record. As Chris correctly guessed, my last patch was based off of the previous patch. This patch puts them together.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5437:
--------------------------------------------
Assignee: Amareshwari Sriramadasu
Status: Patch Available (was: Open)
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Status: Open (was: Patch Available)
This test doesn't detect if jvm reuse is actually working.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5437:
----------------------------------
Status: Open (was: Patch Available)
The diff is relative to trunk + the previous patch
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695417#action_12695417 ]
Hadoop QA commented on HADOOP-5437:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12404552/h5437.patch
against trunk revision 761632.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/110/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/110/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/110/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/110/console
This message is automatically generated.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-5437:
-------------------------------------
Assignee: Owen O'Malley (was: Amareshwari Sriramadasu)
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680416#action_12680416 ]
Hadoop QA commented on HADOOP-5437:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12401736/patch-5437.txt
against trunk revision 751870.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/41/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/41/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/41/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/41/console
This message is automatically generated.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Status: Open (was: Patch Available)
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Status: Patch Available (was: Open)
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Attachment: h5437.patch
This patch:
1. Removes the min split size from the sort run that was effectively forcing 6 maps (and therefore not allowing real jvm reuse).
2. Removes the duplicated set up and tear down the hdfs and mapred cluster.
3. Runs the sort in jvm reuse mode.
4. Adds a new test that uses a static in the mapper to detect whether the jvms are actually reused or not. It tests the reuse flag set to -1 and 1.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5437:
--------------------------------------------
Attachment: patch-5437.txt
Attaching the patch fixing the testcase.
The problem was: The configuration mapred.job.reuse.jvm.num.tasks wa set to -1, but never passed to cluster. Moreover, it should be job configuration, and not cluster configuration, since the configuration is a job-level configuration at tasktarcker.
With the patch I verified jvm reuse is on and each jvm runs multiple tasks.
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-5437:
----------------------------------
Status: Patch Available (was: Open)
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5437) Unit test for jvm -reuse
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12696670#action_12696670 ]
Hudson commented on HADOOP-5437:
--------------------------------
Integrated in Hadoop-trunk #800 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/800/])
> Unit test for jvm -reuse
> ------------------------
>
> Key: HADOOP-5437
> URL: https://issues.apache.org/jira/browse/HADOOP-5437
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: h5437.patch, h5437.patch, h5437.patch, patch-5437.txt
>
>
> TestMiniMRDFSSort.testMapReduceSortWithJvmReuse actually does not test jvm running multiple tasks. In the test, each jvm runs only one task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.