You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Mayank Bansal (JIRA)" <ji...@apache.org> on 2012/07/06 20:40:34 UTC
[jira] [Created] (MAPREDUCE-4403) Adding test case for resubmission
of jobs in TestRecoveryManager
Mayank Bansal created MAPREDUCE-4403:
----------------------------------------
Summary: Adding test case for resubmission of jobs in TestRecoveryManager
Key: MAPREDUCE-4403
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.22.0
Reporter: Mayank Bansal
Assignee: Mayank Bansal
Priority: Minor
In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
There is some refactoring is also needed.
Thanks,
Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4403) Adding test case for
resubmission of jobs in TestRecoveryManager
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411119#comment-13411119 ]
Konstantin Shvachko commented on MAPREDUCE-4403:
------------------------------------------------
The code looks good. Two things:
# {{TEST_DIR = "/tmp";}}
It is better if we don't use /tmp in the tests, as /tmp is not being cleaned up by the build clean target. I suggest to replace it with {{TEST_DIR = "build/tmp";}}
# You need to wrap most of the code into {{try finally}}, where {{finally}} and call {{mr.shutdown()}} in the {{finally}} section. You are starting and stopping JobTracker in the test. When it fails the mini-cluster will not shutdown, which may affect subsequent tests.
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Attachments: MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4403) Adding test case for
resubmission of jobs in TestRecoveryManager
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413539#comment-13413539 ]
Konstantin Shvachko commented on MAPREDUCE-4403:
------------------------------------------------
+1 looks good
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4403) Adding test case for
resubmission of jobs in TestRecoveryManager
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413912#comment-13413912 ]
Mayank Bansal commented on MAPREDUCE-4403:
------------------------------------------
This is not applicable for trunk
Thanks,
Mayank
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Fix For: 0.22.1
>
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4403) Adding test case for resubmission
of jobs in TestRecoveryManager
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mayank Bansal updated MAPREDUCE-4403:
-------------------------------------
Attachment: MAPREDUCE-4403-22.patch
Adding patch for Hadoop 22
Thanks,
Mayank
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Attachments: MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4403) Adding test case for resubmission
of jobs in TestRecoveryManager
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated MAPREDUCE-4403:
-------------------------------------------
Fix Version/s: 0.22.1
Hadoop Flags: Reviewed
I just committed this to branch 0.22.1. Thank you Mayank.
Do we need this for trunk or other versions?
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Fix For: 0.22.1
>
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4403) Adding test case for
resubmission of jobs in TestRecoveryManager
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413743#comment-13413743 ]
Hudson commented on MAPREDUCE-4403:
-----------------------------------
Integrated in Hadoop-Mapreduce-22-branch #109 (See [https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/109/])
MAPREDUCE-4403. Test case for job resubmission in TestRecoveryManager. Contributed by Mayank Bansal. (Revision 1361069)
Result = SUCCESS
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1361069
Files :
* /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt
* /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestRecoveryManager.java
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Fix For: 0.22.1
>
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4403) Adding test case for
resubmission of jobs in TestRecoveryManager
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mayank Bansal resolved MAPREDUCE-4403.
--------------------------------------
Resolution: Fixed
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Fix For: 0.22.1
>
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4403) Adding test case for resubmission
of jobs in TestRecoveryManager
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mayank Bansal updated MAPREDUCE-4403:
-------------------------------------
Attachment: MAPREDUCE-4403-22-1.patch
Hi Konst,
Thanks for the review I updated your comments
Thanks,
Mayank
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4403) Adding test case for resubmission
of jobs in TestRecoveryManager
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mayank Bansal updated MAPREDUCE-4403:
-------------------------------------
Issue Type: Improvement (was: Bug)
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Fix For: 0.22.1
>
> Attachments: MAPREDUCE-4403-22-1.patch, MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MAPREDUCE-4403) Adding test case for
resubmission of jobs in TestRecoveryManager
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411119#comment-13411119 ]
Konstantin Shvachko edited comment on MAPREDUCE-4403 at 7/11/12 6:33 AM:
-------------------------------------------------------------------------
The code looks good. Two things:
# {{TEST_DIR = "/tmp";}}
It is better if we don't use /tmp in the tests, as /tmp is not being cleaned up by the build clean target. I suggest to replace it with {{TEST_DIR = "build/tmp";}}
# You need to wrap most of the code into {{try finally}}, where {{finally}} and call {{mr.shutdown()}} in the {{finally}} section. You are starting and stopping JobTracker in the test. When it fails the mini-cluster will not shutdown, which may affect subsequent tests.
This is not related to your particular patch, because it was introduced in other test cases, and I do not propose to change the old code. I am just advocating to set a better example with the new code.
was (Author: shv):
The code looks good. Two things:
# {{TEST_DIR = "/tmp";}}
It is better if we don't use /tmp in the tests, as /tmp is not being cleaned up by the build clean target. I suggest to replace it with {{TEST_DIR = "build/tmp";}}
# You need to wrap most of the code into {{try finally}}, where {{finally}} and call {{mr.shutdown()}} in the {{finally}} section. You are starting and stopping JobTracker in the test. When it fails the mini-cluster will not shutdown, which may affect subsequent tests.
> Adding test case for resubmission of jobs in TestRecoveryManager
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Priority: Minor
> Attachments: MAPREDUCE-4403-22.patch
>
>
> In Hadoop 22 Test recovery Manager does not have resubmission test case which checks after the resubmission jobs get succeeded.
> There is some refactoring is also needed.
> Thanks,
> Mayank
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira