You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Reka Thirunavukkarasu <re...@wso2.com> on 2014/10/01 14:58:27 UTC
[Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Hi
As you aware, in the composite application we can define the depencies
between groups/cartridges. Autoscaler's responsible is to parse this
dependencies and build up a logical relationship model in order to handle
the dependency information among the child nodes. As we have the
hierarchical monitors in autoscaler, i propose to have dependencies
information in each monitor that they aware of (the immediate child only).
In that monitor, we need to identify the group/cartridge which can be
started in parallel. So that a monitor can look at it's dependency and
control it's immediate children based on that. Once all the children are
active, it can pass the control to it's parent. For Eg:
If we take the top level in Composite application which has mysqlGroup,
postgresGroup, php, tomcat, apimanager and esb. If they have an alias
saying my + cartridge/groupName then we can define the dependency
information as follows:
- myPhp depends on myPostgresGroup (means postgresGroup should
be started before php)
- myTomcat depends on myMysqlGroup
- myApiManager depends on myTomcat
- myEsb depends on myTomcat
Like wise, groups will define their own dependency as well.
In that way, we need to represent these dependency information as part of
Composite Application definition/GroupDefinition. In order to represent
this dependency information given above for Composite Application, i would
suggest to have the following in Composite Application definition.
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
You can use the same format in GroupDefinition to define dependencies in a
group.
As per the example, autoscaler will build a dependency tree for
ApplicationMonitor as below in order to identify the parallel and dependent
ones. So that Autoscaler will start up same level children monitors as in
parallel.
As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
and myMysqlGroup in parallel. Once postgres becomes active,
ApplicationMonitor will start ClusterMonitor for myPhp. Once
myPostgresGroup becomes active, ApplicationMonitor will start the immediate
child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
the myAppServer and myEsb in parallel. This will be applicable for
GroupMonitors as well. They can look at their own dependency tree and will
start their children.
Please share your suggestions on the above model to handle the Dependency
information of Composite Application in autoscaler.
Thanks,
Reka
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Martin,
On Wed, Oct 8, 2014 at 4:21 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Hi Reka,
>
>
>
> In terms of the “Question: in case when we get more than one
> dependencies to be killed, can we kill all of them in parallel or do we
> have to wait until it's dependent cluster/group got killed? “ :
>
>
>
> I checked within our team and it seems that VMs (dependencies) should be
> terminated in the reverse order of the startup sequence. I haven’t heard
> back yet from Shaheed as he is on PTO this week, will follow up as soon as
> I get a response,
>
Thanks for the clarification..If there is any other things to be considered
later, will do it at that moment. I will make it to work as you mentioned
for now..
Thanks,
Reka
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
> *Sent:* Monday, October 06, 2014 10:02 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev; Lakmal Warusawithana; Isuru Haththotuwa; Udara Liyanage
> *Subject:* Re: [Grouping][Part-2] Composite Application Dependency Tree
> building based in Autoscaler
>
>
>
> Hi Martin,
>
>
>
> On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
> Hi all,
>
>
>
> I have implemented the dependency tree as mentioned in my mail earlier. It
> will return the immediate children for the start able dependencies.
>
>
>
> FYI: a composite application has postgresGroup, php, mysqlGroup, app
> server and esb as it's immediate children and their start up order is as
> mentioned in the mail earlier.
>
>
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
>
>
> So, if we look at the kill behaviour of this composite Application, it
> will be like below:
>
>
>
> *kill-none* : none of them will be returned
>
>
>
> *kill-all*: all the elements in that dependency tree will be returned
>
> For eg: if something happened to postgresGroup, then all the children
> of dependency tree would be returned as php, mysqlGroup, app server and esb
> will be get killed.
>
>
>
> *kill-dependent*: all the children of that particular node in the
> dependency tree will be returned.
>
> For eg: If something happened to mysqlGroup, then subsequently tomcat,
> app server and esb would be get killed.
>
>
>
> Question: in case when we get more than one dependencies to be killed, can
> we kill all of them in parallel or do we have to wait until it's dependent
> cluster/group got killed?
>
>
>
> Do you have any input on this?
>
>
>
> Thanks,
>
> Reka
>
>
>
> Thanks,
>
> Reka
>
>
>
> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
> Hi Martin,
>
>
>
> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Reka,
>
>
>
> Are you suggesting to replace the current startupOrder definition with the
> one mentioned below ?
>
>
>
> "startupOrder" : [
>
> {
>
> "start":"aa",
>
> "after":"bb"
>
> }
>
> ]
>
>
>
> Replaced with
>
>
>
> "startupOrders": [
>
> "mypostgresGroup, myphp",
>
> "mysqlGroup, mytomcat",
>
> "mytomcat, myapimanager",
>
> "mytomcat, myesb"
>
> ]
>
>
>
> I have a couple of questions,
>
>
>
> 1. If we use the cartridge alias and the group alias in the group /
> application dependency definition how will it work when we auto scale
> groups ? My current understanding is that to get group scaling to work we
> would need 2 parameters – group name (==group.name) and group instance id
> (== group.alias), one static and one dynamic. So I would think we’ll have
> to define the application dependencies and group dependencies based on the
> name and not the alias, but, during run time we have to calculate the
> dependencies based on the alias.
>
> I think is important to make the distinction between group type (or name,)
> and group instance Id, without it we won’t be able to implement group
> scaling, wdyt ?
>
> Thanks for pointing this out..Yah..As you have mentioned, if we are to
> scale the groups by creating new groups, then we will be unable to use the
> groups alias in place of startuporders. But stratos is tightly coupled with
> subscription to cluster as one to one mapping and also, load balancer uses
> one to one mapping between cluster and hostname. So, if we are to bring up
> new clusters/groups, then things might get complicated in stratos. As i
> explained in the [part-1] discussion, we thought of achieving scale by
> group member and scale by group using constructing the deployment policy in
> a more advanced manner. I will start a separate thread on that. According
> to all of our opinion, we can decide on how to follow that up.
>
> IMHO the startupOrders in composite application and group definitions
> (json ) should look like
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
> while the runtime representation of the logical relationship model for
> each group or cartridge should use the corresponding aliases (or instance
> Id) so the monitor will reference the aliases (or instance Ids) while the
> json application / group definition will reference the group name (or type)
> and cartridge type to define the dependencies, WDYT ?
>
> 2. If for example a cartridge has multiple dependencies we would
> just add another line to the the startupOrders :
>
> e.g. postgresGroup depends on php and abc would be
> represented by:
> "startupOrders": [
> "postgresGroup, php",
> "postgresGroup, “abc”
> ….
> ]
>
>
>
>
>
> Otherwise I think the proposal looks good, +1
>
>
>
> Thanks,
>
> Reka
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
> *Sent:* Wednesday, October 01, 2014 5:58 AM
> *To:* dev
> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
> Udara Liyanage
> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
> building based in Autoscaler
>
>
>
> Hi
>
>
>
> As you aware, in the composite application we can define the depencies
> between groups/cartridges. Autoscaler's responsible is to parse this
> dependencies and build up a logical relationship model in order to handle
> the dependency information among the child nodes. As we have the
> hierarchical monitors in autoscaler, i propose to have dependencies
> information in each monitor that they aware of (the immediate child only).
> In that monitor, we need to identify the group/cartridge which can be
> started in parallel. So that a monitor can look at it's dependency and
> control it's immediate children based on that. Once all the children are
> active, it can pass the control to it's parent. For Eg:
>
>
>
> If we take the top level in Composite application which has mysqlGroup,
> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
> saying my + cartridge/groupName then we can define the dependency
> information as follows:
>
> - myPhp depends on myPostgresGroup (means postgresGroup should
> be started before php)
>
> - myTomcat depends on myMysqlGroup
>
> - myApiManager depends on myTomcat
>
> - myEsb depends on myTomcat
>
>
>
> Like wise, groups will define their own dependency as well.
>
>
>
> In that way, we need to represent these dependency information as part of
> Composite Application definition/GroupDefinition. In order to represent
> this dependency information given above for Composite Application, i would
> suggest to have the following in Composite Application definition.
>
>
>
> "startupOrders": [
>
> "mypostgresGroup, myphp",
>
> "mysqlGroup, mytomcat",
>
> "mytomcat, myapimanager",
>
> "mytomcat, myesb"
>
> ]
>
>
>
> You can use the same format in GroupDefinition to define dependencies in a
> group.
>
>
>
> As per the example, autoscaler will build a dependency tree for
> ApplicationMonitor as below in order to identify the parallel and dependent
> ones. So that Autoscaler will start up same level children monitors as in
> parallel.
>
>
>
>
>
>
> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
> and myMysqlGroup in parallel. Once postgres becomes active,
> ApplicationMonitor will start ClusterMonitor for myPhp. Once
> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
> the myAppServer and myEsb in parallel. This will be applicable for
> GroupMonitors as well. They can look at their own dependency tree and will
> start their children.
>
>
>
> Please share your suggestions on the above model to handle the Dependency
> information of Composite Application in autoscaler.
>
>
>
>
>
> Thanks,
>
> Reka
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
RE: [Grouping][Part-2] Composite Application Dependency Tree
building based in Autoscaler
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Hi Reka,
In terms of the “Question: in case when we get more than one dependencies to be killed, can we kill all of them in parallel or do we have to wait until it's dependent cluster/group got killed? “ :
I checked within our team and it seems that VMs (dependencies) should be terminated in the reverse order of the startup sequence. I haven’t heard back yet from Shaheed as he is on PTO this week, will follow up as soon as I get a response,
Thanks
Martin
From: Reka Thirunavukkarasu [mailto:reka@wso2.com]
Sent: Monday, October 06, 2014 10:02 PM
To: Martin Eppel (meppel)
Cc: dev; Lakmal Warusawithana; Isuru Haththotuwa; Udara Liyanage
Subject: Re: [Grouping][Part-2] Composite Application Dependency Tree building based in Autoscaler
Hi Martin,
On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>> wrote:
Hi all,
I have implemented the dependency tree as mentioned in my mail earlier. It will return the immediate children for the start able dependencies.
FYI: a composite application has postgresGroup, php, mysqlGroup, app server and esb as it's immediate children and their start up order is as mentioned in the mail earlier.
"startupOrders": [
“postgresGroup, php",
"sqlGroup, tomcat",
"tomcat, apimanager",
"tomcat, esb”
]
So, if we look at the kill behaviour of this composite Application, it will be like below:
kill-none : none of them will be returned
kill-all: all the elements in that dependency tree will be returned
For eg: if something happened to postgresGroup, then all the children of dependency tree would be returned as php, mysqlGroup, app server and esb will be get killed.
kill-dependent: all the children of that particular node in the dependency tree will be returned.
For eg: If something happened to mysqlGroup, then subsequently tomcat, app server and esb would be get killed.
Question: in case when we get more than one dependencies to be killed, can we kill all of them in parallel or do we have to wait until it's dependent cluster/group got killed?
Do you have any input on this?
Thanks,
Reka
Thanks,
Reka
On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>> wrote:
Hi Martin,
On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Reka,
Are you suggesting to replace the current startupOrder definition with the one mentioned below ?
"startupOrder" : [
{
"start":"aa",
"after":"bb"
}
]
Replaced with
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
I have a couple of questions,
1. If we use the cartridge alias and the group alias in the group / application dependency definition how will it work when we auto scale groups ? My current understanding is that to get group scaling to work we would need 2 parameters – group name (==group.name<http://group.name>) and group instance id (== group.alias), one static and one dynamic. So I would think we’ll have to define the application dependencies and group dependencies based on the name and not the alias, but, during run time we have to calculate the dependencies based on the alias.
I think is important to make the distinction between group type (or name,) and group instance Id, without it we won’t be able to implement group scaling, wdyt ?
Thanks for pointing this out..Yah..As you have mentioned, if we are to scale the groups by creating new groups, then we will be unable to use the groups alias in place of startuporders. But stratos is tightly coupled with subscription to cluster as one to one mapping and also, load balancer uses one to one mapping between cluster and hostname. So, if we are to bring up new clusters/groups, then things might get complicated in stratos. As i explained in the [part-1] discussion, we thought of achieving scale by group member and scale by group using constructing the deployment policy in a more advanced manner. I will start a separate thread on that. According to all of our opinion, we can decide on how to follow that up.
IMHO the startupOrders in composite application and group definitions (json ) should look like
"startupOrders": [
“postgresGroup, php",
"sqlGroup, tomcat",
"tomcat, apimanager",
"tomcat, esb”
]
while the runtime representation of the logical relationship model for each group or cartridge should use the corresponding aliases (or instance Id) so the monitor will reference the aliases (or instance Ids) while the json application / group definition will reference the group name (or type) and cartridge type to define the dependencies, WDYT ?
2. If for example a cartridge has multiple dependencies we would just add another line to the the startupOrders :
e.g. postgresGroup depends on php and abc would be represented by:
"startupOrders": [
"postgresGroup, php",
"postgresGroup, “abc”
….
]
Otherwise I think the proposal looks good, +1
Thanks,
Reka
Thanks
Martin
From: Reka Thirunavukkarasu [mailto:reka@wso2.com<ma...@wso2.com>]
Sent: Wednesday, October 01, 2014 5:58 AM
To: dev
Cc: Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel); Udara Liyanage
Subject: [Grouping][Part-2] Composite Application Dependency Tree building based in Autoscaler
Hi
As you aware, in the composite application we can define the depencies between groups/cartridges. Autoscaler's responsible is to parse this dependencies and build up a logical relationship model in order to handle the dependency information among the child nodes. As we have the hierarchical monitors in autoscaler, i propose to have dependencies information in each monitor that they aware of (the immediate child only). In that monitor, we need to identify the group/cartridge which can be started in parallel. So that a monitor can look at it's dependency and control it's immediate children based on that. Once all the children are active, it can pass the control to it's parent. For Eg:
If we take the top level in Composite application which has mysqlGroup, postgresGroup, php, tomcat, apimanager and esb. If they have an alias saying my + cartridge/groupName then we can define the dependency information as follows:
- myPhp depends on myPostgresGroup (means postgresGroup should be started before php)
- myTomcat depends on myMysqlGroup
- myApiManager depends on myTomcat
- myEsb depends on myTomcat
Like wise, groups will define their own dependency as well.
In that way, we need to represent these dependency information as part of Composite Application definition/GroupDefinition. In order to represent this dependency information given above for Composite Application, i would suggest to have the following in Composite Application definition.
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
You can use the same format in GroupDefinition to define dependencies in a group.
As per the example, autoscaler will build a dependency tree for ApplicationMonitor as below in order to identify the parallel and dependent ones. So that Autoscaler will start up same level children monitors as in parallel.
[cid:image001.jpg@01CFE224.033D1AF0]
As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup and myMysqlGroup in parallel. Once postgres becomes active, ApplicationMonitor will start ClusterMonitor for myPhp. Once myPostgresGroup becomes active, ApplicationMonitor will start the immediate child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start the myAppServer and myEsb in parallel. This will be applicable for GroupMonitors as well. They can look at their own dependency tree and will start their children.
Please share your suggestions on the above model to handle the Dependency information of Composite Application in autoscaler.
Thanks,
Reka
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Martin,
On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>
wrote:
> Hi all,
>
> I have implemented the dependency tree as mentioned in my mail earlier. It
> will return the immediate children for the start able dependencies.
>
> FYI: a composite application has postgresGroup, php, mysqlGroup, app
> server and esb as it's immediate children and their start up order is as
> mentioned in the mail earlier.
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
> So, if we look at the kill behaviour of this composite Application, it
> will be like below:
>
> *kill-none* : none of them will be returned
>
> *kill-all*: all the elements in that dependency tree will be returned
> For eg: if something happened to postgresGroup, then all the children
> of dependency tree would be returned as php, mysqlGroup, app server and esb
> will be get killed.
>
> *kill-dependent*: all the children of that particular node in the
> dependency tree will be returned.
> For eg: If something happened to mysqlGroup, then subsequently tomcat,
> app server and esb would be get killed.
>
> Question: in case when we get more than one dependencies to be killed, can
> we kill all of them in parallel or do we have to wait until it's dependent
> cluster/group got killed?
>
Do you have any input on this?
Thanks,
Reka
>
> Thanks,
> Reka
>
> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
>> Hi Martin,
>>
>> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>>> Hi Reka,
>>>
>>>
>>>
>>> Are you suggesting to replace the current startupOrder definition with
>>> the one mentioned below ?
>>>
>>>
>>>
>>> "startupOrder" : [
>>>
>>> {
>>>
>>> "start":"aa",
>>>
>>> "after":"bb"
>>>
>>> }
>>>
>>> ]
>>>
>>>
>>>
>>> Replaced with
>>>
>>>
>>>
>>> "startupOrders": [
>>>
>>> "mypostgresGroup, myphp",
>>>
>>> "mysqlGroup, mytomcat",
>>>
>>> "mytomcat, myapimanager",
>>>
>>> "mytomcat, myesb"
>>>
>>> ]
>>>
>>>
>>>
>>> I have a couple of questions,
>>>
>>>
>>>
>>> 1. If we use the cartridge alias and the group alias in the group
>>> / application dependency definition how will it work when we auto scale
>>> groups ? My current understanding is that to get group scaling to work we
>>> would need 2 parameters – group name (==group.name) and group instance
>>> id (== group.alias), one static and one dynamic. So I would think we’ll
>>> have to define the application dependencies and group dependencies based on
>>> the name and not the alias, but, during run time we have to calculate the
>>> dependencies based on the alias.
>>>
>>> I think is important to make the distinction between group type (or
>>> name,) and group instance Id, without it we won’t be able to implement
>>> group scaling, wdyt ?
>>>
>> Thanks for pointing this out..Yah..As you have mentioned, if we are to
>> scale the groups by creating new groups, then we will be unable to use the
>> groups alias in place of startuporders. But stratos is tightly coupled with
>> subscription to cluster as one to one mapping and also, load balancer uses
>> one to one mapping between cluster and hostname. So, if we are to bring up
>> new clusters/groups, then things might get complicated in stratos. As i
>> explained in the [part-1] discussion, we thought of achieving scale by
>> group member and scale by group using constructing the deployment policy in
>> a more advanced manner. I will start a separate thread on that. According
>> to all of our opinion, we can decide on how to follow that up.
>>
>>> IMHO the startupOrders in composite application and group definitions
>>> (json ) should look like
>>>
>>> "startupOrders": [
>>> “postgresGroup, php",
>>> "sqlGroup, tomcat",
>>> "tomcat, apimanager",
>>> "tomcat, esb”
>>> ]
>>>
>>> while the runtime representation of the logical relationship model for
>>> each group or cartridge should use the corresponding aliases (or instance
>>> Id) so the monitor will reference the aliases (or instance Ids) while the
>>> json application / group definition will reference the group name (or type)
>>> and cartridge type to define the dependencies, WDYT ?
>>>
>>> 2. If for example a cartridge has multiple dependencies we would
>>> just add another line to the the startupOrders :
>>>
>>> e.g. postgresGroup depends on php and abc would be
>>> represented by:
>>> "startupOrders": [
>>> "postgresGroup, php",
>>> "postgresGroup, “abc”
>>> ….
>>> ]
>>>
>>>
>>>
>>>
>>>
>>> Otherwise I think the proposal looks good, +1
>>>
>>
>> Thanks,
>> Reka
>>
>>>
>>>
>>> Thanks
>>>
>>>
>>>
>>> Martin
>>>
>>>
>>>
>>> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
>>> *Sent:* Wednesday, October 01, 2014 5:58 AM
>>> *To:* dev
>>> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
>>> Udara Liyanage
>>> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
>>> building based in Autoscaler
>>>
>>>
>>>
>>> Hi
>>>
>>>
>>>
>>> As you aware, in the composite application we can define the depencies
>>> between groups/cartridges. Autoscaler's responsible is to parse this
>>> dependencies and build up a logical relationship model in order to handle
>>> the dependency information among the child nodes. As we have the
>>> hierarchical monitors in autoscaler, i propose to have dependencies
>>> information in each monitor that they aware of (the immediate child only).
>>> In that monitor, we need to identify the group/cartridge which can be
>>> started in parallel. So that a monitor can look at it's dependency and
>>> control it's immediate children based on that. Once all the children are
>>> active, it can pass the control to it's parent. For Eg:
>>>
>>>
>>>
>>> If we take the top level in Composite application which has mysqlGroup,
>>> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
>>> saying my + cartridge/groupName then we can define the dependency
>>> information as follows:
>>>
>>> - myPhp depends on myPostgresGroup (means postgresGroup
>>> should be started before php)
>>>
>>> - myTomcat depends on myMysqlGroup
>>>
>>> - myApiManager depends on myTomcat
>>>
>>> - myEsb depends on myTomcat
>>>
>>>
>>>
>>> Like wise, groups will define their own dependency as well.
>>>
>>>
>>>
>>> In that way, we need to represent these dependency information as part
>>> of Composite Application definition/GroupDefinition. In order to represent
>>> this dependency information given above for Composite Application, i would
>>> suggest to have the following in Composite Application definition.
>>>
>>>
>>>
>>> "startupOrders": [
>>>
>>> "mypostgresGroup, myphp",
>>>
>>> "mysqlGroup, mytomcat",
>>>
>>> "mytomcat, myapimanager",
>>>
>>> "mytomcat, myesb"
>>>
>>> ]
>>>
>>>
>>>
>>> You can use the same format in GroupDefinition to define dependencies in
>>> a group.
>>>
>>>
>>>
>>> As per the example, autoscaler will build a dependency tree for
>>> ApplicationMonitor as below in order to identify the parallel and dependent
>>> ones. So that Autoscaler will start up same level children monitors as in
>>> parallel.
>>>
>>>
>>>
>>>
>>>
>>>
>>> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
>>> and myMysqlGroup in parallel. Once postgres becomes active,
>>> ApplicationMonitor will start ClusterMonitor for myPhp. Once
>>> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
>>> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
>>> the myAppServer and myEsb in parallel. This will be applicable for
>>> GroupMonitors as well. They can look at their own dependency tree and will
>>> start their children.
>>>
>>>
>>>
>>> Please share your suggestions on the above model to handle the
>>> Dependency information of Composite Application in autoscaler.
>>>
>>>
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Reka
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Reka Thirunavukkarasu
>>> Senior Software Engineer,
>>> WSO2, Inc.:http://wso2.com,
>>>
>>> Mobile: +94776442007
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Reka Thirunavukkarasu
>> Senior Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>> Mobile: +94776442007
>>
>>
>>
>
>
> --
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
> Mobile: +94776442007
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Martin,
On Wed, Oct 8, 2014 at 3:56 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Hi Reka,
>
>
>
> Just wanted to clarify, the changes suggested below should completely
> replace the previous structure (StartupOrder with before, after), not only
> in the json definition but also in all subsequent object models, correct ?
>
Yah..We need to change the json, application parser and the Topology.
>
>
> Btw, what triggered this change ?
>
As i think, it is naming issue. If we change it to terminate, then it will
be more consistent with the naming that we currently use in cloud
controller. Or did you mean something else?
Thanks,
Reka
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
> *Sent:* Monday, October 06, 2014 2:22 AM
> *To:* dev
> *Subject:* Re: [Grouping][Part-2] Composite Application Dependency Tree
> building based in Autoscaler
>
>
>
> Hi Imesh,
>
>
>
> On Mon, Oct 6, 2014 at 2:45 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
> Hi Reka,
>
>
>
> I have a small concern on using the term "kill" in this scenario, I think
> it would be much more elegant if we call it something like "terminate".
> WDYT?
>
> +1. It's more appropriate to use terminate. Will change it to terminate.
>
>
>
> Thanks,
>
> Reka
>
>
>
>
>
>
>
> On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
> Hi all,
>
>
>
> I have implemented the dependency tree as mentioned in my mail earlier. It
> will return the immediate children for the start able dependencies.
>
>
>
> FYI: a composite application has postgresGroup, php, mysqlGroup, app
> server and esb as it's immediate children and their start up order is as
> mentioned in the mail earlier.
>
>
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
>
>
> So, if we look at the kill behaviour of this composite Application, it
> will be like below:
>
>
>
> *kill-none* : none of them will be returned
>
>
>
> *kill-all*: all the elements in that dependency tree will be returned
>
> For eg: if something happened to postgresGroup, then all the children
> of dependency tree would be returned as php, mysqlGroup, app server and esb
> will be get killed.
>
>
>
> *kill-dependent*: all the children of that particular node in the
> dependency tree will be returned.
>
> For eg: If something happened to mysqlGroup, then subsequently tomcat,
> app server and esb would be get killed.
>
>
>
> Question: in case when we get more than one dependencies to be killed, can
> we kill all of them in parallel or do we have to wait until it's dependent
> cluster/group got killed?
>
>
>
> Thanks,
>
> Reka
>
>
>
> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
> Hi Martin,
>
>
>
> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Reka,
>
>
>
> Are you suggesting to replace the current startupOrder definition with the
> one mentioned below ?
>
>
>
> "startupOrder" : [
>
> {
>
> "start":"aa",
>
> "after":"bb"
>
> }
>
> ]
>
>
>
> Replaced with
>
>
>
> "startupOrders": [
>
> "mypostgresGroup, myphp",
>
> "mysqlGroup, mytomcat",
>
> "mytomcat, myapimanager",
>
> "mytomcat, myesb"
>
> ]
>
>
>
> I have a couple of questions,
>
>
>
> 1. If we use the cartridge alias and the group alias in the group /
> application dependency definition how will it work when we auto scale
> groups ? My current understanding is that to get group scaling to work we
> would need 2 parameters – group name (==group.name) and group instance id
> (== group.alias), one static and one dynamic. So I would think we’ll have
> to define the application dependencies and group dependencies based on the
> name and not the alias, but, during run time we have to calculate the
> dependencies based on the alias.
>
> I think is important to make the distinction between group type (or name,)
> and group instance Id, without it we won’t be able to implement group
> scaling, wdyt ?
>
> Thanks for pointing this out..Yah..As you have mentioned, if we are to
> scale the groups by creating new groups, then we will be unable to use the
> groups alias in place of startuporders. But stratos is tightly coupled with
> subscription to cluster as one to one mapping and also, load balancer uses
> one to one mapping between cluster and hostname. So, if we are to bring up
> new clusters/groups, then things might get complicated in stratos. As i
> explained in the [part-1] discussion, we thought of achieving scale by
> group member and scale by group using constructing the deployment policy in
> a more advanced manner. I will start a separate thread on that. According
> to all of our opinion, we can decide on how to follow that up.
>
> IMHO the startupOrders in composite application and group definitions
> (json ) should look like
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
> while the runtime representation of the logical relationship model for
> each group or cartridge should use the corresponding aliases (or instance
> Id) so the monitor will reference the aliases (or instance Ids) while the
> json application / group definition will reference the group name (or type)
> and cartridge type to define the dependencies, WDYT ?
>
> 2. If for example a cartridge has multiple dependencies we would
> just add another line to the the startupOrders :
>
> e.g. postgresGroup depends on php and abc would be
> represented by:
> "startupOrders": [
> "postgresGroup, php",
> "postgresGroup, “abc”
> ….
> ]
>
>
>
>
>
> Otherwise I think the proposal looks good, +1
>
>
>
> Thanks,
>
> Reka
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
> *Sent:* Wednesday, October 01, 2014 5:58 AM
> *To:* dev
> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
> Udara Liyanage
> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
> building based in Autoscaler
>
>
>
> Hi
>
>
>
> As you aware, in the composite application we can define the depencies
> between groups/cartridges. Autoscaler's responsible is to parse this
> dependencies and build up a logical relationship model in order to handle
> the dependency information among the child nodes. As we have the
> hierarchical monitors in autoscaler, i propose to have dependencies
> information in each monitor that they aware of (the immediate child only).
> In that monitor, we need to identify the group/cartridge which can be
> started in parallel. So that a monitor can look at it's dependency and
> control it's immediate children based on that. Once all the children are
> active, it can pass the control to it's parent. For Eg:
>
>
>
> If we take the top level in Composite application which has mysqlGroup,
> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
> saying my + cartridge/groupName then we can define the dependency
> information as follows:
>
> - myPhp depends on myPostgresGroup (means postgresGroup should
> be started before php)
>
> - myTomcat depends on myMysqlGroup
>
> - myApiManager depends on myTomcat
>
> - myEsb depends on myTomcat
>
>
>
> Like wise, groups will define their own dependency as well.
>
>
>
> In that way, we need to represent these dependency information as part of
> Composite Application definition/GroupDefinition. In order to represent
> this dependency information given above for Composite Application, i would
> suggest to have the following in Composite Application definition.
>
>
>
> "startupOrders": [
>
> "mypostgresGroup, myphp",
>
> "mysqlGroup, mytomcat",
>
> "mytomcat, myapimanager",
>
> "mytomcat, myesb"
>
> ]
>
>
>
> You can use the same format in GroupDefinition to define dependencies in a
> group.
>
>
>
> As per the example, autoscaler will build a dependency tree for
> ApplicationMonitor as below in order to identify the parallel and dependent
> ones. So that Autoscaler will start up same level children monitors as in
> parallel.
>
>
>
>
>
>
> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
> and myMysqlGroup in parallel. Once postgres becomes active,
> ApplicationMonitor will start ClusterMonitor for myPhp. Once
> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
> the myAppServer and myEsb in parallel. This will be applicable for
> GroupMonitors as well. They can look at their own dependency tree and will
> start their children.
>
>
>
> Please share your suggestions on the above model to handle the Dependency
> information of Composite Application in autoscaler.
>
>
>
>
>
> Thanks,
>
> Reka
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
>
>
>
> --
>
> Imesh Gunaratne
>
>
>
> Technical Lead, WSO2
>
> Committer & PMC Member, Apache Stratos
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
RE: [Grouping][Part-2] Composite Application Dependency Tree
building based in Autoscaler
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Hi Reka,
Just wanted to clarify, the changes suggested below should completely replace the previous structure (StartupOrder with before, after), not only in the json definition but also in all subsequent object models, correct ?
Btw, what triggered this change ?
Thanks
Martin
From: Reka Thirunavukkarasu [mailto:reka@wso2.com]
Sent: Monday, October 06, 2014 2:22 AM
To: dev
Subject: Re: [Grouping][Part-2] Composite Application Dependency Tree building based in Autoscaler
Hi Imesh,
On Mon, Oct 6, 2014 at 2:45 PM, Imesh Gunaratne <im...@apache.org>> wrote:
Hi Reka,
I have a small concern on using the term "kill" in this scenario, I think it would be much more elegant if we call it something like "terminate". WDYT?
+1. It's more appropriate to use terminate. Will change it to terminate.
Thanks,
Reka
On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>> wrote:
Hi all,
I have implemented the dependency tree as mentioned in my mail earlier. It will return the immediate children for the start able dependencies.
FYI: a composite application has postgresGroup, php, mysqlGroup, app server and esb as it's immediate children and their start up order is as mentioned in the mail earlier.
"startupOrders": [
“postgresGroup, php",
"sqlGroup, tomcat",
"tomcat, apimanager",
"tomcat, esb”
]
So, if we look at the kill behaviour of this composite Application, it will be like below:
kill-none : none of them will be returned
kill-all: all the elements in that dependency tree will be returned
For eg: if something happened to postgresGroup, then all the children of dependency tree would be returned as php, mysqlGroup, app server and esb will be get killed.
kill-dependent: all the children of that particular node in the dependency tree will be returned.
For eg: If something happened to mysqlGroup, then subsequently tomcat, app server and esb would be get killed.
Question: in case when we get more than one dependencies to be killed, can we kill all of them in parallel or do we have to wait until it's dependent cluster/group got killed?
Thanks,
Reka
On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>> wrote:
Hi Martin,
On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Reka,
Are you suggesting to replace the current startupOrder definition with the one mentioned below ?
"startupOrder" : [
{
"start":"aa",
"after":"bb"
}
]
Replaced with
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
I have a couple of questions,
1. If we use the cartridge alias and the group alias in the group / application dependency definition how will it work when we auto scale groups ? My current understanding is that to get group scaling to work we would need 2 parameters – group name (==group.name<http://group.name>) and group instance id (== group.alias), one static and one dynamic. So I would think we’ll have to define the application dependencies and group dependencies based on the name and not the alias, but, during run time we have to calculate the dependencies based on the alias.
I think is important to make the distinction between group type (or name,) and group instance Id, without it we won’t be able to implement group scaling, wdyt ?
Thanks for pointing this out..Yah..As you have mentioned, if we are to scale the groups by creating new groups, then we will be unable to use the groups alias in place of startuporders. But stratos is tightly coupled with subscription to cluster as one to one mapping and also, load balancer uses one to one mapping between cluster and hostname. So, if we are to bring up new clusters/groups, then things might get complicated in stratos. As i explained in the [part-1] discussion, we thought of achieving scale by group member and scale by group using constructing the deployment policy in a more advanced manner. I will start a separate thread on that. According to all of our opinion, we can decide on how to follow that up.
IMHO the startupOrders in composite application and group definitions (json ) should look like
"startupOrders": [
“postgresGroup, php",
"sqlGroup, tomcat",
"tomcat, apimanager",
"tomcat, esb”
]
while the runtime representation of the logical relationship model for each group or cartridge should use the corresponding aliases (or instance Id) so the monitor will reference the aliases (or instance Ids) while the json application / group definition will reference the group name (or type) and cartridge type to define the dependencies, WDYT ?
2. If for example a cartridge has multiple dependencies we would just add another line to the the startupOrders :
e.g. postgresGroup depends on php and abc would be represented by:
"startupOrders": [
"postgresGroup, php",
"postgresGroup, “abc”
….
]
Otherwise I think the proposal looks good, +1
Thanks,
Reka
Thanks
Martin
From: Reka Thirunavukkarasu [mailto:reka@wso2.com<ma...@wso2.com>]
Sent: Wednesday, October 01, 2014 5:58 AM
To: dev
Cc: Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel); Udara Liyanage
Subject: [Grouping][Part-2] Composite Application Dependency Tree building based in Autoscaler
Hi
As you aware, in the composite application we can define the depencies between groups/cartridges. Autoscaler's responsible is to parse this dependencies and build up a logical relationship model in order to handle the dependency information among the child nodes. As we have the hierarchical monitors in autoscaler, i propose to have dependencies information in each monitor that they aware of (the immediate child only). In that monitor, we need to identify the group/cartridge which can be started in parallel. So that a monitor can look at it's dependency and control it's immediate children based on that. Once all the children are active, it can pass the control to it's parent. For Eg:
If we take the top level in Composite application which has mysqlGroup, postgresGroup, php, tomcat, apimanager and esb. If they have an alias saying my + cartridge/groupName then we can define the dependency information as follows:
- myPhp depends on myPostgresGroup (means postgresGroup should be started before php)
- myTomcat depends on myMysqlGroup
- myApiManager depends on myTomcat
- myEsb depends on myTomcat
Like wise, groups will define their own dependency as well.
In that way, we need to represent these dependency information as part of Composite Application definition/GroupDefinition. In order to represent this dependency information given above for Composite Application, i would suggest to have the following in Composite Application definition.
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
You can use the same format in GroupDefinition to define dependencies in a group.
As per the example, autoscaler will build a dependency tree for ApplicationMonitor as below in order to identify the parallel and dependent ones. So that Autoscaler will start up same level children monitors as in parallel.
[cid:image001.jpg@01CFE243.0E57E040]
As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup and myMysqlGroup in parallel. Once postgres becomes active, ApplicationMonitor will start ClusterMonitor for myPhp. Once myPostgresGroup becomes active, ApplicationMonitor will start the immediate child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start the myAppServer and myEsb in parallel. This will be applicable for GroupMonitors as well. They can look at their own dependency tree and will start their children.
Please share your suggestions on the above model to handle the Dependency information of Composite Application in autoscaler.
Thanks,
Reka
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Imesh,
On Mon, Oct 6, 2014 at 2:45 PM, Imesh Gunaratne <im...@apache.org> wrote:
> Hi Reka,
>
> I have a small concern on using the term "kill" in this scenario, I think
> it would be much more elegant if we call it something like "terminate".
> WDYT?
>
+1. It's more appropriate to use terminate. Will change it to terminate.
Thanks,
Reka
>
>
>
> On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
>> Hi all,
>>
>> I have implemented the dependency tree as mentioned in my mail earlier.
>> It will return the immediate children for the start able dependencies.
>>
>> FYI: a composite application has postgresGroup, php, mysqlGroup, app
>> server and esb as it's immediate children and their start up order is as
>> mentioned in the mail earlier.
>>
>> "startupOrders": [
>> “postgresGroup, php",
>> "sqlGroup, tomcat",
>> "tomcat, apimanager",
>> "tomcat, esb”
>> ]
>>
>> So, if we look at the kill behaviour of this composite Application, it
>> will be like below:
>>
>> *kill-none* : none of them will be returned
>>
>> *kill-all*: all the elements in that dependency tree will be returned
>> For eg: if something happened to postgresGroup, then all the children
>> of dependency tree would be returned as php, mysqlGroup, app server and esb
>> will be get killed.
>>
>> *kill-dependent*: all the children of that particular node in the
>> dependency tree will be returned.
>> For eg: If something happened to mysqlGroup, then subsequently
>> tomcat, app server and esb would be get killed.
>>
>> Question: in case when we get more than one dependencies to be killed,
>> can we kill all of them in parallel or do we have to wait until it's
>> dependent cluster/group got killed?
>>
>> Thanks,
>> Reka
>>
>> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>
>> wrote:
>>
>>> Hi Martin,
>>>
>>> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <meppel@cisco.com
>>> > wrote:
>>>
>>>> Hi Reka,
>>>>
>>>>
>>>>
>>>> Are you suggesting to replace the current startupOrder definition with
>>>> the one mentioned below ?
>>>>
>>>>
>>>>
>>>> "startupOrder" : [
>>>>
>>>> {
>>>>
>>>> "start":"aa",
>>>>
>>>> "after":"bb"
>>>>
>>>> }
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> Replaced with
>>>>
>>>>
>>>>
>>>> "startupOrders": [
>>>>
>>>> "mypostgresGroup, myphp",
>>>>
>>>> "mysqlGroup, mytomcat",
>>>>
>>>> "mytomcat, myapimanager",
>>>>
>>>> "mytomcat, myesb"
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> I have a couple of questions,
>>>>
>>>>
>>>>
>>>> 1. If we use the cartridge alias and the group alias in the
>>>> group / application dependency definition how will it work when we auto
>>>> scale groups ? My current understanding is that to get group scaling to
>>>> work we would need 2 parameters – group name (==group.name) and group
>>>> instance id (== group.alias), one static and one dynamic. So I would think
>>>> we’ll have to define the application dependencies and group dependencies
>>>> based on the name and not the alias, but, during run time we have to
>>>> calculate the dependencies based on the alias.
>>>>
>>>> I think is important to make the distinction between group type (or
>>>> name,) and group instance Id, without it we won’t be able to implement
>>>> group scaling, wdyt ?
>>>>
>>> Thanks for pointing this out..Yah..As you have mentioned, if we are to
>>> scale the groups by creating new groups, then we will be unable to use the
>>> groups alias in place of startuporders. But stratos is tightly coupled with
>>> subscription to cluster as one to one mapping and also, load balancer uses
>>> one to one mapping between cluster and hostname. So, if we are to bring up
>>> new clusters/groups, then things might get complicated in stratos. As i
>>> explained in the [part-1] discussion, we thought of achieving scale by
>>> group member and scale by group using constructing the deployment policy in
>>> a more advanced manner. I will start a separate thread on that. According
>>> to all of our opinion, we can decide on how to follow that up.
>>>
>>>> IMHO the startupOrders in composite application and group definitions
>>>> (json ) should look like
>>>>
>>>> "startupOrders": [
>>>> “postgresGroup, php",
>>>> "sqlGroup, tomcat",
>>>> "tomcat, apimanager",
>>>> "tomcat, esb”
>>>> ]
>>>>
>>>> while the runtime representation of the logical relationship model for
>>>> each group or cartridge should use the corresponding aliases (or instance
>>>> Id) so the monitor will reference the aliases (or instance Ids) while the
>>>> json application / group definition will reference the group name (or type)
>>>> and cartridge type to define the dependencies, WDYT ?
>>>>
>>>> 2. If for example a cartridge has multiple dependencies we would
>>>> just add another line to the the startupOrders :
>>>>
>>>> e.g. postgresGroup depends on php and abc would be
>>>> represented by:
>>>> "startupOrders": [
>>>> "postgresGroup, php",
>>>> "postgresGroup, “abc”
>>>> ….
>>>> ]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Otherwise I think the proposal looks good, +1
>>>>
>>>
>>> Thanks,
>>> Reka
>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> Martin
>>>>
>>>>
>>>>
>>>> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
>>>> *Sent:* Wednesday, October 01, 2014 5:58 AM
>>>> *To:* dev
>>>> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
>>>> Udara Liyanage
>>>> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
>>>> building based in Autoscaler
>>>>
>>>>
>>>>
>>>> Hi
>>>>
>>>>
>>>>
>>>> As you aware, in the composite application we can define the depencies
>>>> between groups/cartridges. Autoscaler's responsible is to parse this
>>>> dependencies and build up a logical relationship model in order to handle
>>>> the dependency information among the child nodes. As we have the
>>>> hierarchical monitors in autoscaler, i propose to have dependencies
>>>> information in each monitor that they aware of (the immediate child only).
>>>> In that monitor, we need to identify the group/cartridge which can be
>>>> started in parallel. So that a monitor can look at it's dependency and
>>>> control it's immediate children based on that. Once all the children are
>>>> active, it can pass the control to it's parent. For Eg:
>>>>
>>>>
>>>>
>>>> If we take the top level in Composite application which has mysqlGroup,
>>>> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
>>>> saying my + cartridge/groupName then we can define the dependency
>>>> information as follows:
>>>>
>>>> - myPhp depends on myPostgresGroup (means postgresGroup
>>>> should be started before php)
>>>>
>>>> - myTomcat depends on myMysqlGroup
>>>>
>>>> - myApiManager depends on myTomcat
>>>>
>>>> - myEsb depends on myTomcat
>>>>
>>>>
>>>>
>>>> Like wise, groups will define their own dependency as well.
>>>>
>>>>
>>>>
>>>> In that way, we need to represent these dependency information as part
>>>> of Composite Application definition/GroupDefinition. In order to represent
>>>> this dependency information given above for Composite Application, i would
>>>> suggest to have the following in Composite Application definition.
>>>>
>>>>
>>>>
>>>> "startupOrders": [
>>>>
>>>> "mypostgresGroup, myphp",
>>>>
>>>> "mysqlGroup, mytomcat",
>>>>
>>>> "mytomcat, myapimanager",
>>>>
>>>> "mytomcat, myesb"
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> You can use the same format in GroupDefinition to define dependencies
>>>> in a group.
>>>>
>>>>
>>>>
>>>> As per the example, autoscaler will build a dependency tree for
>>>> ApplicationMonitor as below in order to identify the parallel and dependent
>>>> ones. So that Autoscaler will start up same level children monitors as in
>>>> parallel.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> As above, ApplicationMonitor will start GroupMonitors of
>>>> myPostgresGroup and myMysqlGroup in parallel. Once postgres becomes active,
>>>> ApplicationMonitor will start ClusterMonitor for myPhp. Once
>>>> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
>>>> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
>>>> the myAppServer and myEsb in parallel. This will be applicable for
>>>> GroupMonitors as well. They can look at their own dependency tree and will
>>>> start their children.
>>>>
>>>>
>>>>
>>>> Please share your suggestions on the above model to handle the
>>>> Dependency information of Composite Application in autoscaler.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Reka
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Reka Thirunavukkarasu
>>>> Senior Software Engineer,
>>>> WSO2, Inc.:http://wso2.com,
>>>>
>>>> Mobile: +94776442007
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Reka Thirunavukkarasu
>>> Senior Software Engineer,
>>> WSO2, Inc.:http://wso2.com,
>>> Mobile: +94776442007
>>>
>>>
>>>
>>
>>
>> --
>> Reka Thirunavukkarasu
>> Senior Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>> Mobile: +94776442007
>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Lakmal Warusawithana <la...@wso2.com>.
On Mon, Oct 6, 2014 at 2:45 PM, Imesh Gunaratne <im...@apache.org> wrote:
> Hi Reka,
>
> I have a small concern on using the term "kill" in this scenario, I think
> it would be much more elegant if we call it something like "terminate".
> WDYT?
>
+1 for terminate.
>
> Thanks
>
> On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
>> Hi all,
>>
>> I have implemented the dependency tree as mentioned in my mail earlier.
>> It will return the immediate children for the start able dependencies.
>>
>> FYI: a composite application has postgresGroup, php, mysqlGroup, app
>> server and esb as it's immediate children and their start up order is as
>> mentioned in the mail earlier.
>>
>> "startupOrders": [
>> “postgresGroup, php",
>> "sqlGroup, tomcat",
>> "tomcat, apimanager",
>> "tomcat, esb”
>> ]
>>
>> So, if we look at the kill behaviour of this composite Application, it
>> will be like below:
>>
>> *kill-none* : none of them will be returned
>>
>> *kill-all*: all the elements in that dependency tree will be returned
>> For eg: if something happened to postgresGroup, then all the children
>> of dependency tree would be returned as php, mysqlGroup, app server and esb
>> will be get killed.
>>
>> *kill-dependent*: all the children of that particular node in the
>> dependency tree will be returned.
>> For eg: If something happened to mysqlGroup, then subsequently
>> tomcat, app server and esb would be get killed.
>>
>> Question: in case when we get more than one dependencies to be killed,
>> can we kill all of them in parallel or do we have to wait until it's
>> dependent cluster/group got killed?
>>
>> Thanks,
>> Reka
>>
>> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>
>> wrote:
>>
>>> Hi Martin,
>>>
>>> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <meppel@cisco.com
>>> > wrote:
>>>
>>>> Hi Reka,
>>>>
>>>>
>>>>
>>>> Are you suggesting to replace the current startupOrder definition with
>>>> the one mentioned below ?
>>>>
>>>>
>>>>
>>>> "startupOrder" : [
>>>>
>>>> {
>>>>
>>>> "start":"aa",
>>>>
>>>> "after":"bb"
>>>>
>>>> }
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> Replaced with
>>>>
>>>>
>>>>
>>>> "startupOrders": [
>>>>
>>>> "mypostgresGroup, myphp",
>>>>
>>>> "mysqlGroup, mytomcat",
>>>>
>>>> "mytomcat, myapimanager",
>>>>
>>>> "mytomcat, myesb"
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> I have a couple of questions,
>>>>
>>>>
>>>>
>>>> 1. If we use the cartridge alias and the group alias in the
>>>> group / application dependency definition how will it work when we auto
>>>> scale groups ? My current understanding is that to get group scaling to
>>>> work we would need 2 parameters – group name (==group.name) and group
>>>> instance id (== group.alias), one static and one dynamic. So I would think
>>>> we’ll have to define the application dependencies and group dependencies
>>>> based on the name and not the alias, but, during run time we have to
>>>> calculate the dependencies based on the alias.
>>>>
>>>> I think is important to make the distinction between group type (or
>>>> name,) and group instance Id, without it we won’t be able to implement
>>>> group scaling, wdyt ?
>>>>
>>> Thanks for pointing this out..Yah..As you have mentioned, if we are to
>>> scale the groups by creating new groups, then we will be unable to use the
>>> groups alias in place of startuporders. But stratos is tightly coupled with
>>> subscription to cluster as one to one mapping and also, load balancer uses
>>> one to one mapping between cluster and hostname. So, if we are to bring up
>>> new clusters/groups, then things might get complicated in stratos. As i
>>> explained in the [part-1] discussion, we thought of achieving scale by
>>> group member and scale by group using constructing the deployment policy in
>>> a more advanced manner. I will start a separate thread on that. According
>>> to all of our opinion, we can decide on how to follow that up.
>>>
>>>> IMHO the startupOrders in composite application and group definitions
>>>> (json ) should look like
>>>>
>>>> "startupOrders": [
>>>> “postgresGroup, php",
>>>> "sqlGroup, tomcat",
>>>> "tomcat, apimanager",
>>>> "tomcat, esb”
>>>> ]
>>>>
>>>> while the runtime representation of the logical relationship model for
>>>> each group or cartridge should use the corresponding aliases (or instance
>>>> Id) so the monitor will reference the aliases (or instance Ids) while the
>>>> json application / group definition will reference the group name (or type)
>>>> and cartridge type to define the dependencies, WDYT ?
>>>>
>>>> 2. If for example a cartridge has multiple dependencies we would
>>>> just add another line to the the startupOrders :
>>>>
>>>> e.g. postgresGroup depends on php and abc would be
>>>> represented by:
>>>> "startupOrders": [
>>>> "postgresGroup, php",
>>>> "postgresGroup, “abc”
>>>> ….
>>>> ]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Otherwise I think the proposal looks good, +1
>>>>
>>>
>>> Thanks,
>>> Reka
>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> Martin
>>>>
>>>>
>>>>
>>>> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
>>>> *Sent:* Wednesday, October 01, 2014 5:58 AM
>>>> *To:* dev
>>>> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
>>>> Udara Liyanage
>>>> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
>>>> building based in Autoscaler
>>>>
>>>>
>>>>
>>>> Hi
>>>>
>>>>
>>>>
>>>> As you aware, in the composite application we can define the depencies
>>>> between groups/cartridges. Autoscaler's responsible is to parse this
>>>> dependencies and build up a logical relationship model in order to handle
>>>> the dependency information among the child nodes. As we have the
>>>> hierarchical monitors in autoscaler, i propose to have dependencies
>>>> information in each monitor that they aware of (the immediate child only).
>>>> In that monitor, we need to identify the group/cartridge which can be
>>>> started in parallel. So that a monitor can look at it's dependency and
>>>> control it's immediate children based on that. Once all the children are
>>>> active, it can pass the control to it's parent. For Eg:
>>>>
>>>>
>>>>
>>>> If we take the top level in Composite application which has mysqlGroup,
>>>> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
>>>> saying my + cartridge/groupName then we can define the dependency
>>>> information as follows:
>>>>
>>>> - myPhp depends on myPostgresGroup (means postgresGroup
>>>> should be started before php)
>>>>
>>>> - myTomcat depends on myMysqlGroup
>>>>
>>>> - myApiManager depends on myTomcat
>>>>
>>>> - myEsb depends on myTomcat
>>>>
>>>>
>>>>
>>>> Like wise, groups will define their own dependency as well.
>>>>
>>>>
>>>>
>>>> In that way, we need to represent these dependency information as part
>>>> of Composite Application definition/GroupDefinition. In order to represent
>>>> this dependency information given above for Composite Application, i would
>>>> suggest to have the following in Composite Application definition.
>>>>
>>>>
>>>>
>>>> "startupOrders": [
>>>>
>>>> "mypostgresGroup, myphp",
>>>>
>>>> "mysqlGroup, mytomcat",
>>>>
>>>> "mytomcat, myapimanager",
>>>>
>>>> "mytomcat, myesb"
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> You can use the same format in GroupDefinition to define dependencies
>>>> in a group.
>>>>
>>>>
>>>>
>>>> As per the example, autoscaler will build a dependency tree for
>>>> ApplicationMonitor as below in order to identify the parallel and dependent
>>>> ones. So that Autoscaler will start up same level children monitors as in
>>>> parallel.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> As above, ApplicationMonitor will start GroupMonitors of
>>>> myPostgresGroup and myMysqlGroup in parallel. Once postgres becomes active,
>>>> ApplicationMonitor will start ClusterMonitor for myPhp. Once
>>>> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
>>>> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
>>>> the myAppServer and myEsb in parallel. This will be applicable for
>>>> GroupMonitors as well. They can look at their own dependency tree and will
>>>> start their children.
>>>>
>>>>
>>>>
>>>> Please share your suggestions on the above model to handle the
>>>> Dependency information of Composite Application in autoscaler.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Reka
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Reka Thirunavukkarasu
>>>> Senior Software Engineer,
>>>> WSO2, Inc.:http://wso2.com,
>>>>
>>>> Mobile: +94776442007
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Reka Thirunavukkarasu
>>> Senior Software Engineer,
>>> WSO2, Inc.:http://wso2.com,
>>> Mobile: +94776442007
>>>
>>>
>>>
>>
>>
>> --
>> Reka Thirunavukkarasu
>> Senior Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>> Mobile: +94776442007
>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Imesh Gunaratne <im...@apache.org>.
Hi Reka,
I have a small concern on using the term "kill" in this scenario, I think
it would be much more elegant if we call it something like "terminate".
WDYT?
Thanks
On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <re...@wso2.com>
wrote:
> Hi all,
>
> I have implemented the dependency tree as mentioned in my mail earlier. It
> will return the immediate children for the start able dependencies.
>
> FYI: a composite application has postgresGroup, php, mysqlGroup, app
> server and esb as it's immediate children and their start up order is as
> mentioned in the mail earlier.
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
> So, if we look at the kill behaviour of this composite Application, it
> will be like below:
>
> *kill-none* : none of them will be returned
>
> *kill-all*: all the elements in that dependency tree will be returned
> For eg: if something happened to postgresGroup, then all the children
> of dependency tree would be returned as php, mysqlGroup, app server and esb
> will be get killed.
>
> *kill-dependent*: all the children of that particular node in the
> dependency tree will be returned.
> For eg: If something happened to mysqlGroup, then subsequently tomcat,
> app server and esb would be get killed.
>
> Question: in case when we get more than one dependencies to be killed, can
> we kill all of them in parallel or do we have to wait until it's dependent
> cluster/group got killed?
>
> Thanks,
> Reka
>
> On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
>> Hi Martin,
>>
>> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>>> Hi Reka,
>>>
>>>
>>>
>>> Are you suggesting to replace the current startupOrder definition with
>>> the one mentioned below ?
>>>
>>>
>>>
>>> "startupOrder" : [
>>>
>>> {
>>>
>>> "start":"aa",
>>>
>>> "after":"bb"
>>>
>>> }
>>>
>>> ]
>>>
>>>
>>>
>>> Replaced with
>>>
>>>
>>>
>>> "startupOrders": [
>>>
>>> "mypostgresGroup, myphp",
>>>
>>> "mysqlGroup, mytomcat",
>>>
>>> "mytomcat, myapimanager",
>>>
>>> "mytomcat, myesb"
>>>
>>> ]
>>>
>>>
>>>
>>> I have a couple of questions,
>>>
>>>
>>>
>>> 1. If we use the cartridge alias and the group alias in the group
>>> / application dependency definition how will it work when we auto scale
>>> groups ? My current understanding is that to get group scaling to work we
>>> would need 2 parameters – group name (==group.name) and group instance
>>> id (== group.alias), one static and one dynamic. So I would think we’ll
>>> have to define the application dependencies and group dependencies based on
>>> the name and not the alias, but, during run time we have to calculate the
>>> dependencies based on the alias.
>>>
>>> I think is important to make the distinction between group type (or
>>> name,) and group instance Id, without it we won’t be able to implement
>>> group scaling, wdyt ?
>>>
>> Thanks for pointing this out..Yah..As you have mentioned, if we are to
>> scale the groups by creating new groups, then we will be unable to use the
>> groups alias in place of startuporders. But stratos is tightly coupled with
>> subscription to cluster as one to one mapping and also, load balancer uses
>> one to one mapping between cluster and hostname. So, if we are to bring up
>> new clusters/groups, then things might get complicated in stratos. As i
>> explained in the [part-1] discussion, we thought of achieving scale by
>> group member and scale by group using constructing the deployment policy in
>> a more advanced manner. I will start a separate thread on that. According
>> to all of our opinion, we can decide on how to follow that up.
>>
>>> IMHO the startupOrders in composite application and group definitions
>>> (json ) should look like
>>>
>>> "startupOrders": [
>>> “postgresGroup, php",
>>> "sqlGroup, tomcat",
>>> "tomcat, apimanager",
>>> "tomcat, esb”
>>> ]
>>>
>>> while the runtime representation of the logical relationship model for
>>> each group or cartridge should use the corresponding aliases (or instance
>>> Id) so the monitor will reference the aliases (or instance Ids) while the
>>> json application / group definition will reference the group name (or type)
>>> and cartridge type to define the dependencies, WDYT ?
>>>
>>> 2. If for example a cartridge has multiple dependencies we would
>>> just add another line to the the startupOrders :
>>>
>>> e.g. postgresGroup depends on php and abc would be
>>> represented by:
>>> "startupOrders": [
>>> "postgresGroup, php",
>>> "postgresGroup, “abc”
>>> ….
>>> ]
>>>
>>>
>>>
>>>
>>>
>>> Otherwise I think the proposal looks good, +1
>>>
>>
>> Thanks,
>> Reka
>>
>>>
>>>
>>> Thanks
>>>
>>>
>>>
>>> Martin
>>>
>>>
>>>
>>> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
>>> *Sent:* Wednesday, October 01, 2014 5:58 AM
>>> *To:* dev
>>> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
>>> Udara Liyanage
>>> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
>>> building based in Autoscaler
>>>
>>>
>>>
>>> Hi
>>>
>>>
>>>
>>> As you aware, in the composite application we can define the depencies
>>> between groups/cartridges. Autoscaler's responsible is to parse this
>>> dependencies and build up a logical relationship model in order to handle
>>> the dependency information among the child nodes. As we have the
>>> hierarchical monitors in autoscaler, i propose to have dependencies
>>> information in each monitor that they aware of (the immediate child only).
>>> In that monitor, we need to identify the group/cartridge which can be
>>> started in parallel. So that a monitor can look at it's dependency and
>>> control it's immediate children based on that. Once all the children are
>>> active, it can pass the control to it's parent. For Eg:
>>>
>>>
>>>
>>> If we take the top level in Composite application which has mysqlGroup,
>>> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
>>> saying my + cartridge/groupName then we can define the dependency
>>> information as follows:
>>>
>>> - myPhp depends on myPostgresGroup (means postgresGroup
>>> should be started before php)
>>>
>>> - myTomcat depends on myMysqlGroup
>>>
>>> - myApiManager depends on myTomcat
>>>
>>> - myEsb depends on myTomcat
>>>
>>>
>>>
>>> Like wise, groups will define their own dependency as well.
>>>
>>>
>>>
>>> In that way, we need to represent these dependency information as part
>>> of Composite Application definition/GroupDefinition. In order to represent
>>> this dependency information given above for Composite Application, i would
>>> suggest to have the following in Composite Application definition.
>>>
>>>
>>>
>>> "startupOrders": [
>>>
>>> "mypostgresGroup, myphp",
>>>
>>> "mysqlGroup, mytomcat",
>>>
>>> "mytomcat, myapimanager",
>>>
>>> "mytomcat, myesb"
>>>
>>> ]
>>>
>>>
>>>
>>> You can use the same format in GroupDefinition to define dependencies in
>>> a group.
>>>
>>>
>>>
>>> As per the example, autoscaler will build a dependency tree for
>>> ApplicationMonitor as below in order to identify the parallel and dependent
>>> ones. So that Autoscaler will start up same level children monitors as in
>>> parallel.
>>>
>>>
>>>
>>>
>>>
>>>
>>> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
>>> and myMysqlGroup in parallel. Once postgres becomes active,
>>> ApplicationMonitor will start ClusterMonitor for myPhp. Once
>>> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
>>> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
>>> the myAppServer and myEsb in parallel. This will be applicable for
>>> GroupMonitors as well. They can look at their own dependency tree and will
>>> start their children.
>>>
>>>
>>>
>>> Please share your suggestions on the above model to handle the
>>> Dependency information of Composite Application in autoscaler.
>>>
>>>
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Reka
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Reka Thirunavukkarasu
>>> Senior Software Engineer,
>>> WSO2, Inc.:http://wso2.com,
>>>
>>> Mobile: +94776442007
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Reka Thirunavukkarasu
>> Senior Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>> Mobile: +94776442007
>>
>>
>>
>
>
> --
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
> Mobile: +94776442007
>
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi all,
I have implemented the dependency tree as mentioned in my mail earlier. It
will return the immediate children for the start able dependencies.
FYI: a composite application has postgresGroup, php, mysqlGroup, app
server and esb as it's immediate children and their start up order is as
mentioned in the mail earlier.
"startupOrders": [
“postgresGroup, php",
"sqlGroup, tomcat",
"tomcat, apimanager",
"tomcat, esb”
]
So, if we look at the kill behaviour of this composite Application, it will
be like below:
*kill-none* : none of them will be returned
*kill-all*: all the elements in that dependency tree will be returned
For eg: if something happened to postgresGroup, then all the children
of dependency tree would be returned as php, mysqlGroup, app server and esb
will be get killed.
*kill-dependent*: all the children of that particular node in the
dependency tree will be returned.
For eg: If something happened to mysqlGroup, then subsequently tomcat,
app server and esb would be get killed.
Question: in case when we get more than one dependencies to be killed, can
we kill all of them in parallel or do we have to wait until it's dependent
cluster/group got killed?
Thanks,
Reka
On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <re...@wso2.com> wrote:
> Hi Martin,
>
> On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
>> Hi Reka,
>>
>>
>>
>> Are you suggesting to replace the current startupOrder definition with
>> the one mentioned below ?
>>
>>
>>
>> "startupOrder" : [
>>
>> {
>>
>> "start":"aa",
>>
>> "after":"bb"
>>
>> }
>>
>> ]
>>
>>
>>
>> Replaced with
>>
>>
>>
>> "startupOrders": [
>>
>> "mypostgresGroup, myphp",
>>
>> "mysqlGroup, mytomcat",
>>
>> "mytomcat, myapimanager",
>>
>> "mytomcat, myesb"
>>
>> ]
>>
>>
>>
>> I have a couple of questions,
>>
>>
>>
>> 1. If we use the cartridge alias and the group alias in the group
>> / application dependency definition how will it work when we auto scale
>> groups ? My current understanding is that to get group scaling to work we
>> would need 2 parameters – group name (==group.name) and group instance
>> id (== group.alias), one static and one dynamic. So I would think we’ll
>> have to define the application dependencies and group dependencies based on
>> the name and not the alias, but, during run time we have to calculate the
>> dependencies based on the alias.
>>
>> I think is important to make the distinction between group type (or
>> name,) and group instance Id, without it we won’t be able to implement
>> group scaling, wdyt ?
>>
> Thanks for pointing this out..Yah..As you have mentioned, if we are to
> scale the groups by creating new groups, then we will be unable to use the
> groups alias in place of startuporders. But stratos is tightly coupled with
> subscription to cluster as one to one mapping and also, load balancer uses
> one to one mapping between cluster and hostname. So, if we are to bring up
> new clusters/groups, then things might get complicated in stratos. As i
> explained in the [part-1] discussion, we thought of achieving scale by
> group member and scale by group using constructing the deployment policy in
> a more advanced manner. I will start a separate thread on that. According
> to all of our opinion, we can decide on how to follow that up.
>
>> IMHO the startupOrders in composite application and group definitions
>> (json ) should look like
>>
>> "startupOrders": [
>> “postgresGroup, php",
>> "sqlGroup, tomcat",
>> "tomcat, apimanager",
>> "tomcat, esb”
>> ]
>>
>> while the runtime representation of the logical relationship model for
>> each group or cartridge should use the corresponding aliases (or instance
>> Id) so the monitor will reference the aliases (or instance Ids) while the
>> json application / group definition will reference the group name (or type)
>> and cartridge type to define the dependencies, WDYT ?
>>
>> 2. If for example a cartridge has multiple dependencies we would
>> just add another line to the the startupOrders :
>>
>> e.g. postgresGroup depends on php and abc would be
>> represented by:
>> "startupOrders": [
>> "postgresGroup, php",
>> "postgresGroup, “abc”
>> ….
>> ]
>>
>>
>>
>>
>>
>> Otherwise I think the proposal looks good, +1
>>
>
> Thanks,
> Reka
>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
>> *Sent:* Wednesday, October 01, 2014 5:58 AM
>> *To:* dev
>> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
>> Udara Liyanage
>> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
>> building based in Autoscaler
>>
>>
>>
>> Hi
>>
>>
>>
>> As you aware, in the composite application we can define the depencies
>> between groups/cartridges. Autoscaler's responsible is to parse this
>> dependencies and build up a logical relationship model in order to handle
>> the dependency information among the child nodes. As we have the
>> hierarchical monitors in autoscaler, i propose to have dependencies
>> information in each monitor that they aware of (the immediate child only).
>> In that monitor, we need to identify the group/cartridge which can be
>> started in parallel. So that a monitor can look at it's dependency and
>> control it's immediate children based on that. Once all the children are
>> active, it can pass the control to it's parent. For Eg:
>>
>>
>>
>> If we take the top level in Composite application which has mysqlGroup,
>> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
>> saying my + cartridge/groupName then we can define the dependency
>> information as follows:
>>
>> - myPhp depends on myPostgresGroup (means postgresGroup
>> should be started before php)
>>
>> - myTomcat depends on myMysqlGroup
>>
>> - myApiManager depends on myTomcat
>>
>> - myEsb depends on myTomcat
>>
>>
>>
>> Like wise, groups will define their own dependency as well.
>>
>>
>>
>> In that way, we need to represent these dependency information as part of
>> Composite Application definition/GroupDefinition. In order to represent
>> this dependency information given above for Composite Application, i would
>> suggest to have the following in Composite Application definition.
>>
>>
>>
>> "startupOrders": [
>>
>> "mypostgresGroup, myphp",
>>
>> "mysqlGroup, mytomcat",
>>
>> "mytomcat, myapimanager",
>>
>> "mytomcat, myesb"
>>
>> ]
>>
>>
>>
>> You can use the same format in GroupDefinition to define dependencies in
>> a group.
>>
>>
>>
>> As per the example, autoscaler will build a dependency tree for
>> ApplicationMonitor as below in order to identify the parallel and dependent
>> ones. So that Autoscaler will start up same level children monitors as in
>> parallel.
>>
>>
>>
>>
>>
>>
>> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
>> and myMysqlGroup in parallel. Once postgres becomes active,
>> ApplicationMonitor will start ClusterMonitor for myPhp. Once
>> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
>> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
>> the myAppServer and myEsb in parallel. This will be applicable for
>> GroupMonitors as well. They can look at their own dependency tree and will
>> start their children.
>>
>>
>>
>> Please share your suggestions on the above model to handle the Dependency
>> information of Composite Application in autoscaler.
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Reka
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Reka Thirunavukkarasu
>> Senior Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>>
>> Mobile: +94776442007
>>
>>
>>
>>
>>
>
>
>
> --
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
> Mobile: +94776442007
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Grouping][Part-2] Composite Application Dependency Tree building
based in Autoscaler
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Martin,
On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Hi Reka,
>
>
>
> Are you suggesting to replace the current startupOrder definition with the
> one mentioned below ?
>
>
>
> "startupOrder" : [
>
> {
>
> "start":"aa",
>
> "after":"bb"
>
> }
>
> ]
>
>
>
> Replaced with
>
>
>
> "startupOrders": [
>
> "mypostgresGroup, myphp",
>
> "mysqlGroup, mytomcat",
>
> "mytomcat, myapimanager",
>
> "mytomcat, myesb"
>
> ]
>
>
>
> I have a couple of questions,
>
>
>
> 1. If we use the cartridge alias and the group alias in the group /
> application dependency definition how will it work when we auto scale
> groups ? My current understanding is that to get group scaling to work we
> would need 2 parameters – group name (==group.name) and group instance id
> (== group.alias), one static and one dynamic. So I would think we’ll have
> to define the application dependencies and group dependencies based on the
> name and not the alias, but, during run time we have to calculate the
> dependencies based on the alias.
>
> I think is important to make the distinction between group type (or name,)
> and group instance Id, without it we won’t be able to implement group
> scaling, wdyt ?
>
Thanks for pointing this out..Yah..As you have mentioned, if we are to
scale the groups by creating new groups, then we will be unable to use the
groups alias in place of startuporders. But stratos is tightly coupled with
subscription to cluster as one to one mapping and also, load balancer uses
one to one mapping between cluster and hostname. So, if we are to bring up
new clusters/groups, then things might get complicated in stratos. As i
explained in the [part-1] discussion, we thought of achieving scale by
group member and scale by group using constructing the deployment policy in
a more advanced manner. I will start a separate thread on that. According
to all of our opinion, we can decide on how to follow that up.
> IMHO the startupOrders in composite application and group definitions
> (json ) should look like
>
> "startupOrders": [
> “postgresGroup, php",
> "sqlGroup, tomcat",
> "tomcat, apimanager",
> "tomcat, esb”
> ]
>
> while the runtime representation of the logical relationship model for
> each group or cartridge should use the corresponding aliases (or instance
> Id) so the monitor will reference the aliases (or instance Ids) while the
> json application / group definition will reference the group name (or type)
> and cartridge type to define the dependencies, WDYT ?
>
> 2. If for example a cartridge has multiple dependencies we would
> just add another line to the the startupOrders :
>
> e.g. postgresGroup depends on php and abc would be
> represented by:
> "startupOrders": [
> "postgresGroup, php",
> "postgresGroup, “abc”
> ….
> ]
>
>
>
>
>
> Otherwise I think the proposal looks good, +1
>
Thanks,
Reka
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
> *Sent:* Wednesday, October 01, 2014 5:58 AM
> *To:* dev
> *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel);
> Udara Liyanage
> *Subject:* [Grouping][Part-2] Composite Application Dependency Tree
> building based in Autoscaler
>
>
>
> Hi
>
>
>
> As you aware, in the composite application we can define the depencies
> between groups/cartridges. Autoscaler's responsible is to parse this
> dependencies and build up a logical relationship model in order to handle
> the dependency information among the child nodes. As we have the
> hierarchical monitors in autoscaler, i propose to have dependencies
> information in each monitor that they aware of (the immediate child only).
> In that monitor, we need to identify the group/cartridge which can be
> started in parallel. So that a monitor can look at it's dependency and
> control it's immediate children based on that. Once all the children are
> active, it can pass the control to it's parent. For Eg:
>
>
>
> If we take the top level in Composite application which has mysqlGroup,
> postgresGroup, php, tomcat, apimanager and esb. If they have an alias
> saying my + cartridge/groupName then we can define the dependency
> information as follows:
>
> - myPhp depends on myPostgresGroup (means postgresGroup should
> be started before php)
>
> - myTomcat depends on myMysqlGroup
>
> - myApiManager depends on myTomcat
>
> - myEsb depends on myTomcat
>
>
>
> Like wise, groups will define their own dependency as well.
>
>
>
> In that way, we need to represent these dependency information as part of
> Composite Application definition/GroupDefinition. In order to represent
> this dependency information given above for Composite Application, i would
> suggest to have the following in Composite Application definition.
>
>
>
> "startupOrders": [
>
> "mypostgresGroup, myphp",
>
> "mysqlGroup, mytomcat",
>
> "mytomcat, myapimanager",
>
> "mytomcat, myesb"
>
> ]
>
>
>
> You can use the same format in GroupDefinition to define dependencies in a
> group.
>
>
>
> As per the example, autoscaler will build a dependency tree for
> ApplicationMonitor as below in order to identify the parallel and dependent
> ones. So that Autoscaler will start up same level children monitors as in
> parallel.
>
>
>
>
>
>
> As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup
> and myMysqlGroup in parallel. Once postgres becomes active,
> ApplicationMonitor will start ClusterMonitor for myPhp. Once
> myPostgresGroup becomes active, ApplicationMonitor will start the immediate
> child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start
> the myAppServer and myEsb in parallel. This will be applicable for
> GroupMonitors as well. They can look at their own dependency tree and will
> start their children.
>
>
>
> Please share your suggestions on the above model to handle the Dependency
> information of Composite Application in autoscaler.
>
>
>
>
>
> Thanks,
>
> Reka
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
RE: [Grouping][Part-2] Composite Application Dependency Tree
building based in Autoscaler
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Hi Reka,
Are you suggesting to replace the current startupOrder definition with the one mentioned below ?
"startupOrder" : [
{
"start":"aa",
"after":"bb"
}
]
Replaced with
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
I have a couple of questions,
1. If we use the cartridge alias and the group alias in the group / application dependency definition how will it work when we auto scale groups ? My current understanding is that to get group scaling to work we would need 2 parameters – group name (==group.name) and group instance id (== group.alias), one static and one dynamic. So I would think we’ll have to define the application dependencies and group dependencies based on the name and not the alias, but, during run time we have to calculate the dependencies based on the alias.
I think is important to make the distinction between group type (or name,) and group instance Id, without it we won’t be able to implement group scaling, wdyt ?
IMHO the startupOrders in composite application and group definitions (json ) should look like
"startupOrders": [
“postgresGroup, php",
"sqlGroup, tomcat",
"tomcat, apimanager",
"tomcat, esb”
]
while the runtime representation of the logical relationship model for each group or cartridge should use the corresponding aliases (or instance Id) so the monitor will reference the aliases (or instance Ids) while the json application / group definition will reference the group name (or type) and cartridge type to define the dependencies, WDYT ?
2. If for example a cartridge has multiple dependencies we would just add another line to the the startupOrders :
e.g. postgresGroup depends on php and abc would be represented by:
"startupOrders": [
"postgresGroup, php",
"postgresGroup, “abc”
….
]
Otherwise I think the proposal looks good, +1
Thanks
Martin
From: Reka Thirunavukkarasu [mailto:reka@wso2.com]
Sent: Wednesday, October 01, 2014 5:58 AM
To: dev
Cc: Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel); Udara Liyanage
Subject: [Grouping][Part-2] Composite Application Dependency Tree building based in Autoscaler
Hi
As you aware, in the composite application we can define the depencies between groups/cartridges. Autoscaler's responsible is to parse this dependencies and build up a logical relationship model in order to handle the dependency information among the child nodes. As we have the hierarchical monitors in autoscaler, i propose to have dependencies information in each monitor that they aware of (the immediate child only). In that monitor, we need to identify the group/cartridge which can be started in parallel. So that a monitor can look at it's dependency and control it's immediate children based on that. Once all the children are active, it can pass the control to it's parent. For Eg:
If we take the top level in Composite application which has mysqlGroup, postgresGroup, php, tomcat, apimanager and esb. If they have an alias saying my + cartridge/groupName then we can define the dependency information as follows:
- myPhp depends on myPostgresGroup (means postgresGroup should be started before php)
- myTomcat depends on myMysqlGroup
- myApiManager depends on myTomcat
- myEsb depends on myTomcat
Like wise, groups will define their own dependency as well.
In that way, we need to represent these dependency information as part of Composite Application definition/GroupDefinition. In order to represent this dependency information given above for Composite Application, i would suggest to have the following in Composite Application definition.
"startupOrders": [
"mypostgresGroup, myphp",
"mysqlGroup, mytomcat",
"mytomcat, myapimanager",
"mytomcat, myesb"
]
You can use the same format in GroupDefinition to define dependencies in a group.
As per the example, autoscaler will build a dependency tree for ApplicationMonitor as below in order to identify the parallel and dependent ones. So that Autoscaler will start up same level children monitors as in parallel.
[cid:image002.jpg@01CFDD5F.FC78A3B0]
As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup and myMysqlGroup in parallel. Once postgres becomes active, ApplicationMonitor will start ClusterMonitor for myPhp. Once myPostgresGroup becomes active, ApplicationMonitor will start the immediate child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start the myAppServer and myEsb in parallel. This will be applicable for GroupMonitors as well. They can look at their own dependency tree and will start their children.
Please share your suggestions on the above model to handle the Dependency information of Composite Application in autoscaler.
Thanks,
Reka
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007<tel:%2B94776442007>