You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2010/01/08 09:14:54 UTC
[jira] Created: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
---------------------------------------------------------------------------
Key: MAPREDUCE-1366
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Reporter: Amareshwari Sriramadasu
Fix For: 0.22.0
Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1366:
-------------------------------------
Status: Patch Available (was: Open)
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799091#action_12799091 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1366:
----------------------------------------------------
Thanks Chris for uploading a patch.
When i tried to test the patch, I realized that the test timeout on MAPREDUCE-1365 is because of MAPREDUCE-1371.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1366:
-----------------------------------------------
Attachment: M1366-1.patch
Patch is updated to trunk and removes the return value from checkFatalError()
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch, M1366-1.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1366:
-----------------------------------------------
Status: Patch Available (was: Open)
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch, M1366-1.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1366:
-------------------------------------
Status: Open (was: Patch Available)
Failures appear related to MAPREDUCE-1275. Will try again.
bq. When i tried to test the patch, I realized that the test timeout on MAPREDUCE-1365 is because of MAPREDUCE-1371.
*nod* Yes, you're right. I hadn't tested that. The test timeout wasn't my motivation, but the spurious failure in MAPREDUCE-64 that would be easier to diagnose.
bq. I would only set the fatalError value if it is not null, so that the earliest fault gets retained. A setFatalError() method could do this.
I don't see what you mean. Each tracker retains its cause of death; it's not shared between them and each tracker should only set this once. Are you suggesting making the error global and retaining only the first fault across all trackers?
bq. Also, this may be an opportunity to give the MiniMRCluster and MinDFS cluster a common base class rather than continue to duplicate code.
Refactoring the Mini\*Clusters is out of scope for this issue. This is just making the cause of test failures related to HADOOP-4744 clearer.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893583#action_12893583 ]
Hadoop QA commented on MAPREDUCE-1366:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12450801/M1366-1.patch
against trunk revision 980316.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/337/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/337/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/337/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/337/console
This message is automatically generated.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch, M1366-1.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799162#action_12799162 ]
Hadoop QA commented on MAPREDUCE-1366:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12429994/M1366-0.patch
against trunk revision 898019.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/377/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/377/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/377/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/377/console
This message is automatically generated.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1366:
-----------------------------------------------
Status: Open (was: Patch Available)
Though patch applies, it does not merge hunks correctly.
Also, return value from JobTrackerRunner.checkFatalError is never used.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799499#action_12799499 ]
Hadoop QA commented on MAPREDUCE-1366:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12429994/M1366-0.patch
against trunk revision 898486.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/267/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/267/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/267/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/267/console
This message is automatically generated.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1366:
-------------------------------------
Attachment: M1366-0.patch
Modified MiniMRCluster to throw {{RuntimeException}} if JobTracker/TaskTracker startup/shutdown fails.
Tested the patch by inserting {{IllegalArgumentException}} throws in JT/TT init, offerService, and shutdown. In each instance, the test failed with the appropriate error.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1366:
-------------------------------------
Status: Patch Available (was: Open)
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1366:
-----------------------------------------------
Status: Open (was: Patch Available)
Test failure TestMapredSystemDir should be fixed
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch, M1366-1.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1366) Tests should not timeout if
TaskTracker/JobTracker crashes in MiniMRCluster
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799188#action_12799188 ]
Steve Loughran commented on MAPREDUCE-1366:
-------------------------------------------
I would only set the fatalError value if it is not null, so that the earliest fault gets retained. A setFatalError() method could do this. Also, this may be an opportunity to give the MiniMRCluster and MinDFS cluster a common base class rather than continue to duplicate code.
> Tests should not timeout if TaskTracker/JobTracker crashes in MiniMRCluster
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-1366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: M1366-0.patch
>
>
> Currently tests timeout if there is any problem bringing up JobTracker or TaskTracker in MiniMRCluster. Instead tests should fail saying JT/TT crashed.
> See test timeout on MAPREDUCE-1365
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.