[jira] [Commented] (WHIRR-221) Optionally control the order of starting services

["Bruno Dumon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/WHIRR-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066060#comment-13066060 ] 

Bruno Dumon commented on WHIRR-221:
-----------------------------------

I thought about implementing this in the context of WHIRR-334, but finally was able to use another solution there (polling for HDFS availability before starting HBase).

I have been thinking a bit about the following: supposing all services are able to wait until the services on which they depend are available, is there still a need for ordered service startup?

I think there might, for example: the HBase master waits at startup for region servers to appear, until their number remained stable in the (by default) last 4.5s (see ServerManager.waitForRegionServers) (you can also configure HBase to wait for a minimum number of region servers to be available at startup). I'm not sure about the details, but I assume that after this time it will start reassigning the regions of unavailable region servers. On a fresh cluster, there won't be any regions yet, so this would seem like unimportant. However, I'm planning to install my own service using Whirr which at startup creates tables in HBase with initial region splits. If not all region servers are available yet, HBase will afterwards have to rebalance those regions (and possibly the same for the corresponding hdfs files).

I've seen some of the configure scripts do a sleep after starting a service (e.g. the hadoop datanode), which is also a way of stating that order is important. If startup order would be controlled by Whirr, these kinds of sleeps (or more intelligent checks of service availability such as a port check) between startup of dependent roles could be handled at that level, which is easier to maintain.

To make things robust for larger clusters, we should not impose an absolute startup order but rather wait for a minimum % of started processes for some role.

Regardless of startup order, it would be a good thing if Whirr had knowledge of the dependencies between service roles and also of roles that should have only one instance. This would allow to validate the cluster template early on.

> Optionally control the order of starting services
> -------------------------------------------------
>
>                 Key: WHIRR-221
>                 URL: https://issues.apache.org/jira/browse/WHIRR-221
>             Project: Whirr
>          Issue Type: New Feature
>          Components: core, documentation
>            Reporter: Andrei Savu
>            Assignee: Andrei Savu
>            Priority: Critical
>         Attachments: WHIRR-221-v1.patch, WHIRR-221-v3.patch, WHIRR-221.patch
>
>
> As Lars sugested in WHIRR-170:
> The user should "be able to optionally control the order (services start). This could be role based and specified like so
> {code}
> whirr.role-order=zk,nn+jt,dn+tt,hbase-master,hbase-regionserver
> {code}
> If not specified the system should make any effort to start the services as quickly as possible, for example in multiple threads. In other words, when the role-order is not given no guarantee about order can be given."

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira