You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Hyunsik Choi (JIRA)" <ji...@apache.org> on 2011/03/19 09:47:29 UTC
[jira] Created: (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Separate tasks of both install and configuration scripts into more generic and detailed functions
-------------------------------------------------------------------------------------------------
Key: WHIRR-266
URL: https://issues.apache.org/jira/browse/WHIRR-266
Project: Whirr
Issue Type: New Feature
Components: core
Reporter: Hyunsik Choi
Fix For: 0.5.0
Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executed individually by a user.
I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-266:
------------------------------
Attachment: WHIRR-266.patch
Simple patch for ZooKeeper tested with byon on Ubuntu. Should we do something similar for all services?
We should open new JIRAs for handling rolling restarts, re-configuration, role cleanup etc.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-266:
----------------------------
Priority: Major (was: Blocker)
Fix Version/s: (was: 0.5.0)
Andrei pointed out that this isn't a blocker for 0.5.0 since not all the services need to be converted. Since I'm having some problems getting the HBase service converted (some kind of race with the thrift server) I'm removing this as a blocker.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-266:
------------------------------
Assignee: (was: Andrei Savu)
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-266:
------------------------------
Attachment: WHIRR-266.patch
Updated scripts for ZooKeeper and elasticsearch. Guys I need your help to update the other services (we can create sub-tasks).
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch, WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-266:
------------------------------
Attachment: (was: WHIRR-266.patch)
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026795#comment-13026795 ]
Hyunsik Choi commented on WHIRR-266:
------------------------------------
These functions (e.g., install, cleanup, service-start, and service-stop) need to be executable individually by user. Therefore, additional CLI commands are necessary. What do you think about that?
> It's not going to be easy to get rid of them in a way that's compatible with multiple OSes. Do you think it's a good idea to add a process supervisor that should work the same on all platforms? (e.g. supervisord)
For independence of multiple OSs, it would be good to immediately use scripts (e.g zkServer.sh or hadoop-daemon.sh) that services provide. I think that service-start/stop scripts can serve as scripts in /etc/init.d/.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020370#comment-13020370 ]
Tom White commented on WHIRR-266:
---------------------------------
This would help development of new services too. See WHIRR-285.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executed individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027717#comment-13027717 ]
Hyunsik Choi commented on WHIRR-266:
------------------------------------
> I believe we should add more CLI commands in a new JIRA and for now focus only on separating tasks.
You are right :) We should do that.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028787#comment-13028787 ]
Andrei Savu commented on WHIRR-266:
-----------------------------------
I've created sub-tasks for all needed changes and moved the attached patch.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026347#comment-13026347 ]
Andrei Savu commented on WHIRR-266:
-----------------------------------
> The current patch seems to have some Ubuntu/Debian-isms
It's not going to be easy to get rid of them in a way that's compatible with multiple OSes.
Do you think it's a good idea to add a process supervisor that should work the same on all platforms? (e.g. supervisord)
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023343#comment-13023343 ]
Andrei Savu commented on WHIRR-266:
-----------------------------------
This should also allow us to do configuration changes and rolling restarts.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu reassigned WHIRR-266:
---------------------------------
Assignee: Andrei Savu
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi updated WHIRR-266:
-------------------------------
Description:
Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
was:
Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executed individually by a user.
I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023497#comment-13023497 ]
Hyunsik Choi commented on WHIRR-266:
------------------------------------
How fine should we separate generic functions from install and configuration scripts? Above all, I would like to discuss this issue.
In general, most services may need the following steps:
* launch instances
* fetch a service program from specific URLs or local tarballs
* install the service program (if necessary, it can be divided to unpack and copy)
* configure the service by user configuration
* launch services
* stop services
* cleaning the service program (remove the service program and configuration)
* stop instances
If we combine them, implementing new features (e.g., restarting service, adding additional cluster nodes, and reusing cluster nodes) and developing new services would be easy.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027617#comment-13027617 ]
Andrei Savu commented on WHIRR-266:
-----------------------------------
> Therefore, additional CLI commands are necessary. What do you think about that?
I believe we should add more CLI commands in a new JIRA and for now focus only on separating tasks.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Assignee: Andrei Savu
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-266) Separate tasks of both install and
configuration scripts into more generic and detailed functions
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025926#comment-13025926 ]
Tom White commented on WHIRR-266:
---------------------------------
> Simple patch for ZooKeeper tested with byon on Ubuntu.
The current patch seems to have some Ubuntu/Debian-isms (update-rc.d isn't used by Redhat, I believe), so it needs updating before it can be committed. Good to hear byon being used!
> Should we do something similar for all services?
Yes, I think this is the right direction, particularly if we have WHIRR-173. The stop and cleanup functions are relevant to WHIRR-239 too.
> Separate tasks of both install and configuration scripts into more generic and detailed functions
> -------------------------------------------------------------------------------------------------
>
> Key: WHIRR-266
> URL: https://issues.apache.org/jira/browse/WHIRR-266
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Reporter: Hyunsik Choi
> Fix For: 0.5.0
>
> Attachments: WHIRR-266.patch
>
>
> Using whirr, I experimented some problems. Especially, if at least one of the steps for installing and configuring a service meets some exception or error, whirr cannot construct the running cluster. In another case, some of entire clusters do not startup by whirr due to unknown reasons. In both cases, users have to recover them manually. It may be really burden for users.
> However, the current implementation does not support any recovery ways. In addition, service scripts (install and post-configure) bundles up too many tasks. For example, for hadoop service the configuration script mounts devices, configures *-site.xml files, and executes the service start script. It makes users hard to reuse that scripts.
> Thus, I propose that we separate both scripts (i.e., install and post-configure) into more normalized and detailed functions, such as *service-clean, service-install, service-configuration, service-start, and service-stop*. In addition, they should be executable individually by a user.
> I expect that it will give us many benefits: 1) it makes debugging easier 2) when an exception or error occurs during starting a service, a user can recover easily the cluster by using appropriate functions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira