You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2008/09/03 12:21:44 UTC
[jira] Created: (HADOOP-4059) Add tests that try starting the
various hadoop command line scripts
Add tests that try starting the various hadoop command line scripts
-------------------------------------------------------------------
Key: HADOOP-4059
URL: https://issues.apache.org/jira/browse/HADOOP-4059
Project: Hadoop Core
Issue Type: Test
Components: test
Affects Versions: 0.19.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Hadoop has lots of tests that start clusters, but I don't see any that test the command line scripts working. Which means that changes to the scripts and the code behind them may not get picked up rapidly, and regressions won't get picked up by Hudson and apportioned to the specific patches.
Propose:
* an abstract test case that can exec scripts on startup; wait for them to finish, kill them if they take too long
* test cases for every service (namenode, tasktracker, datanode, etc)
* tests that try invalid commands, -help options
* tests that start the services, assert that they are live, and shut them down
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4059) Add tests that try starting the
various hadoop command line scripts
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628169#action_12628169 ]
Steve Loughran commented on HADOOP-4059:
----------------------------------------
I hadnt looked at that before..yes, there is some good code there to drive it off XML files not java code. But the current tests dont run the scripts to start/stop cluster services, just the FS operations. I need tests for the services. testCLI seems a good starting point for some of this.
Incidentally, I've worked on past projects where the XML Files generated a different test case for junit; there is a trick to doing this (static TestSuite method), which then results in separate test reports for every command in the file. We could use this trick again.
> Add tests that try starting the various hadoop command line scripts
> -------------------------------------------------------------------
>
> Key: HADOOP-4059
> URL: https://issues.apache.org/jira/browse/HADOOP-4059
> Project: Hadoop Core
> Issue Type: Test
> Components: test
> Affects Versions: 0.19.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Hadoop has lots of tests that start clusters, but I don't see any that test the command line scripts working. Which means that changes to the scripts and the code behind them may not get picked up rapidly, and regressions won't get picked up by Hudson and apportioned to the specific patches.
> Propose:
> * an abstract test case that can exec scripts on startup; wait for them to finish, kill them if they take too long
> * test cases for every service (namenode, tasktracker, datanode, etc)
> * tests that try invalid commands, -help options
> * tests that start the services, assert that they are live, and shut them down
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4059) Add tests that try starting the
various hadoop command line scripts
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627959#action_12627959 ]
Steve Loughran commented on HADOOP-4059:
----------------------------------------
This may actually be easiest to test using AntUnit, as ant has much of the process management code needed: http://ant.apache.org/antlibs/antunit/
It could be added as a contribs/ section that has nothing but tests that get run in a series of test build files. The various entry points do have to run in their own process (we can't call JobTracker.main() etc) as the tests really do have to check the scripts end-to-end. Go via main() in a sandbox to catch System.exit() and you aren't testing the scripts any more, which always create more trouble than 50 lines of code really should
> Add tests that try starting the various hadoop command line scripts
> -------------------------------------------------------------------
>
> Key: HADOOP-4059
> URL: https://issues.apache.org/jira/browse/HADOOP-4059
> Project: Hadoop Core
> Issue Type: Test
> Components: test
> Affects Versions: 0.19.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Hadoop has lots of tests that start clusters, but I don't see any that test the command line scripts working. Which means that changes to the scripts and the code behind them may not get picked up rapidly, and regressions won't get picked up by Hudson and apportioned to the specific patches.
> Propose:
> * an abstract test case that can exec scripts on startup; wait for them to finish, kill them if they take too long
> * test cases for every service (namenode, tasktracker, datanode, etc)
> * tests that try invalid commands, -help options
> * tests that start the services, assert that they are live, and shut them down
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4059) Add tests that try starting the
various hadoop command line scripts
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628039#action_12628039 ]
Nigel Daley commented on HADOOP-4059:
-------------------------------------
FYI, I think org.apache.hadoop.cli.TestCLI tests some of this.
> Add tests that try starting the various hadoop command line scripts
> -------------------------------------------------------------------
>
> Key: HADOOP-4059
> URL: https://issues.apache.org/jira/browse/HADOOP-4059
> Project: Hadoop Core
> Issue Type: Test
> Components: test
> Affects Versions: 0.19.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Hadoop has lots of tests that start clusters, but I don't see any that test the command line scripts working. Which means that changes to the scripts and the code behind them may not get picked up rapidly, and regressions won't get picked up by Hudson and apportioned to the specific patches.
> Propose:
> * an abstract test case that can exec scripts on startup; wait for them to finish, kill them if they take too long
> * test cases for every service (namenode, tasktracker, datanode, etc)
> * tests that try invalid commands, -help options
> * tests that start the services, assert that they are live, and shut them down
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.