You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Andrei Savu (JIRA)" <ji...@apache.org> on 2011/05/04 16:52:03 UTC
[jira] [Created] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Separate Hadoop install and configuration scripts into more generic functions
------------------------------------------------------------------------------
Key: WHIRR-294
URL: https://issues.apache.org/jira/browse/WHIRR-294
Project: Whirr
Issue Type: Sub-task
Components: service/hadoop
Reporter: Andrei Savu
Separate Hadoop install and configuration scripts into more generic functions.
We should have something like:
{start,stop,install,configure,cleanup}_hadoop
Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033084#comment-13033084 ]
Tom White commented on WHIRR-294:
---------------------------------
The namenode start is doing some work to set up directories after the daemon has started, so it needs to stay there, although it should be made idempotent. The namenode format operation could be moved out, but it's only called once so is not really a problem in practice. The CDH start does some package installation - I'll move this to the install script.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White reassigned WHIRR-294:
-------------------------------
Assignee: Tom White
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Attachment: WHIRR-294.patch
Synced with trunk. I'd like to commit this later unless there are any comments or objections.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Attachment: WHIRR-294.patch
Thanks for taking a look, Andrei. Here's a new patch with an HBase start function separated out. Not quite ready for commit as I'm also having problems with the HBase integration test.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Attachment: WHIRR-294.patch
Here's an untested patch for Hadoop (including CDH). It includes a method on ClusterActionHandlerSupport for enforcing naming consistency of functions.
I haven't included stop or cleanup functions since they don't exist currently (so are not called by anything), and can be added in a follow on JIRA.
I still need to run integration tests for this.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Attachment: WHIRR-294.patch
Here's a patch which makes the changes in my previous comment. Ran Hadoop integration tests successfully.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Priority: Major (was: Blocker)
Fix Version/s: (was: 0.5.0)
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033069#comment-13033069 ]
Andrei Savu commented on WHIRR-294:
-----------------------------------
Looks good but I believe that some of the start functions are doing a bit too much (e.g. start_namenode is doing setup work, cdh start_hadoop_daemon installing services). We should move that code to the install / configure functions so that we can reuse the start / stop functions when updating the configuration or doing rolling restarts.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Status: Patch Available (was: Open)
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Attachment: WHIRR-294.patch
Updated patch for Hadoop and CDH which passes integration tests on EC2 and Rackspace.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-294:
----------------------------
Attachment: WHIRR-294.patch
Regenerated patch.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-294:
------------------------------
Fix Version/s: 0.7.0
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.7.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046073#comment-13046073 ]
Tom White commented on WHIRR-294:
---------------------------------
(Not ready for commit yet - still needs testing.)
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-294) Separate Hadoop install and
configuration scripts into more generic functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033837#comment-13033837 ]
Andrei Savu commented on WHIRR-294:
-----------------------------------
Looks good but I'm unable to run the CDH Hbase integration test. This could be only a local (connection) error.
> Separate Hadoop install and configuration scripts into more generic functions
> ------------------------------------------------------------------------------
>
> Key: WHIRR-294
> URL: https://issues.apache.org/jira/browse/WHIRR-294
> Project: Whirr
> Issue Type: Sub-task
> Components: service/hadoop
> Reporter: Andrei Savu
> Assignee: Tom White
> Fix For: 0.5.0
>
> Attachments: WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch, WHIRR-294.patch
>
>
> Separate Hadoop install and configuration scripts into more generic functions.
> We should have something like:
> {start,stop,install,configure,cleanup}_hadoop
> Naming consistency would be helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira