You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by "Martin Eppel (meppel)" <me...@cisco.com> on 2014/06/03 22:51:38 UTC
adding the groupId to a member event ... RE: [Discuss] Grouping of
services (cartridges)
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com [mailto:isuruh@wso2.com] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Will be sending it to you directly
From: isuruh@wso2.com [mailto:isuruh@wso2.com] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, June 04, 2014 12:05 PM
To: dev@stratos.apache.org
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Can you send the locally built Cloud Controller jar (with your changes), so that we can try to generate the WSDL.
On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I patched it but still see the same issue, what’s the best way to debug this ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 11:20 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
I remember you faced a similar issue earlier. So, just to make sure :-)
On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I recompiled all jars, generated the zips and installed the system from the installer zip files, you think it is still necessary ?
(but I’ll try anyway - recompile service stub and patch)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 10:58 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Isuru Haththotuwa <is...@apache.org>.
Great to hear that Martin!
On Thu, Jun 5, 2014 at 8:16 AM, Nirmal Fernando <ni...@gmail.com>
wrote:
> Awesome!
>
>
> On Thu, Jun 5, 2014 at 8:01 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
>> It worked,
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 4:03 PM
>>
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jun 5, 2014 at 3:24 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Ok, I see – I think I don’t understand the correct process to add a new
>> interface to the CloudController.
>>
>>
>>
>> Is this the correct procedure to develop the SOAP API for the could
>> controller : ?
>>
>>
>>
>> 1. Implement the new API in CloudControllerServiceImpl
>>
>> 2. Recompile cloud controller project and patch the server
>>
>> 3. Retrieve the wsdl (Cloudcontroller.wsdl) and place it in my
>> build environment
>>
>> 3.2 place it in the corresponding service stub and replace the existing
>> wsdl under src/main/resurces and build the project.
>>
>> 4. Implement the API in CloudControllerServiceClient, recompile
>> all projects and redeploy binaries
>>
>> Correct.
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 2:34 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> That is a risky thing to do :-) I suggest you to start the product with
>> your patch and then access the auto-generated wsdl using
>> https://<host>:<port>/services/CloudControllerService?wsdl
>>
>>
>>
>> On Thu, Jun 5, 2014 at 1:31 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> I edited my changes manually …
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 12:16 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> How did you generate the WSDL, Martin?
>>
>>
>>
>> On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>> Hi Martin,
>>
>> Can you send the locally built Cloud Controller jar (with your changes),
>> so that we can try to generate the WSDL.
>>
>>
>>
>> On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> I patched it but still see the same issue, what’s the best way to debug
>> this ?
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 11:20 AM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> I remember you faced a similar issue earlier. So, just to make sure :-)
>>
>>
>>
>> On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> I recompiled all jars, generated the zips and installed the system from
>> the installer zip files, you think it is still necessary ?
>>
>> (but I’ll try anyway - recompile service stub and patch)
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 10:58 AM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Can you try to patch the CC service stub to the running product and check?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Here we go, see attached files:
>>
>>
>>
>> ServiceUtils invokes the API
>>
>> CloudControllerClient, CloudControllerServiceImpl for implementation
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 9:45 AM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
>> wrote:
>>
>> I am sorry Martin, but it'll easy for me, if you can attach the code diff
>> of CC component.
>>
>> Thanks.
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Attached to the email,
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 10:20 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Can you please send a diff over?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> No, but I modeled / followed the deployCartridgeDefinition API which I
>> think passes a bean as parameter ?
>>
>>
>>
>> Is it required to define a bean ? Where and how ?
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 8:57 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Did you introduce a new bean class?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Actually,
>>
>>
>>
>> Below are all the changes I made to the wsdl for the new API, all I want
>> at this point is to pass a string parameter in the API:
>>
>>
>>
>> <wsdl:operation name="deployCompositeApplicationDefinition">
>>
>> <soap:operation
>> soapAction="urn:deployCompositeApplicationDefinition"
>> style="document"></soap:operation>
>>
>> <wsdl:input>
>>
>> <soap:body use="literal"></soap:body>
>>
>> </wsdl:input>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <soap:fault use="literal"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>>
>> </wsdl:fault>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidIaasProviderException">
>>
>> <soap:fault use="literal"
>> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>>
>> </wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>> <wsdl:operation name="deployCompositeApplicationDefinition">
>>
>> <soap12:operation
>> soapAction="urn:deployCompositeApplicationDefinition"
>> style="document"></soap12:operation>
>>
>> <wsdl:input>
>>
>> <soap12:body use="literal"></soap12:body>
>>
>> </wsdl:input>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <soap12:fault use="literal"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>>
>> </wsdl:fault>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidIaasProviderException">
>>
>> <soap12:fault use="literal"
>> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>>
>> </wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>> <wsdl:operation
>> name="deployCompositeApplicationDefinition">
>>
>> <http:operation
>> location="deployCompositeApplicationDefinition"></http:operation>
>>
>> <wsdl:input>
>>
>> <mime:content type="text/xml"
>> part="parameters"></mime:content>
>>
>> </wsdl:input>
>>
>> </wsdl:operation>
>>
>>
>>
>>
>>
>> <xs:element
>> name="deployCompositeApplicationDefinition">
>>
>> <xs:complexType>
>>
>> <xs:sequence>
>>
>> <xs:element minOccurs="0"
>> name="compositeApplicationConfig" nillable="true"
>> type="xs:string"></xs:element>
>>
>> </xs:sequence>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>>
>>
>>
>>
>>
>>
>> <wsdl:message
>> name="deployCompositeApplicationDefinitionRequest">
>>
>> <wsdl:part name="parameters"
>> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>>
>> </wsdl:message>
>>
>>
>>
>> <wsdl:operation
>> name="deployCompositeApplicationDefinition">
>>
>> <wsdl:input
>> message="ns:deployCompositeApplicationDefinitionRequest"
>> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>>
>> <wsdl:fault
>> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
>> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>>
>> <wsdl:fault
>> message="ns:CloudControllerServiceInvalidIaasProviderException"
>> name="CloudControllerServiceInvalidIaasProviderException"
>> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>>
>>
>> <xs:element
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <xs:complexType>
>>
>> <xs:sequence>
>>
>> <xs:element minOccurs="0"
>> name="InvalidCompositeApplicationDefinitionException" nillable="true"
>> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>>
>> </xs:sequence>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>>
>>
>> <wsdl:message
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <wsdl:part name="parameters"
>> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>>
>> </wsdl:message>
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Tuesday, June 03, 2014 8:47 PM
>> *To:* dev@stratos.apache.org
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Yes,
>>
>>
>>
>> I did add the new api in the cloudcontroller.wsdl and regenerated the
>> stub (recompiled successfully all projects, including service-stub, etc …).
>> The error happens during run time when I invoke the API, are there any
>> debug traces I can turn on for axis which might help ?
>>
>>
>>
>>
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Tuesday, June 03, 2014 8:11 PM
>> *To:* dev@stratos.apache.org
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
>> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Marin,
>>
>> If you have changed the CloudController interface, you will need to use
>> the new cloud controller WSDL and re-generate the Cloud Controller service
>> stub code. This error can be a result of this. Let me know if you have not
>> done this so that I can give some guide lines.
>>
>>
>>
>> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> When I invoke my soap API I get the following exception, what ‘s the best
>> way to troublshoot / debug this ? What could be the issue ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
>> occurred while trying to invoke service method
>> deployCompositeApplicationDefinition
>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>>
>> java.lang.ArrayIndexOutOfBoundsException: 1
>>
>> at
>> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>>
>> at
>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>>
>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>
>> at
>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>>
>> at
>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>>
>> at
>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>
>> at
>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>
>> at
>> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>
>> at
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>
>> at
>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>
>> at
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>
>> at
>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>
>> at
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>
>> at
>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
>> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
>> occurred while trying to invoke service method
>> deployCompositeApplicationDefinition
>> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>>
>> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
>> service method deployCompositeApplicationDefinition
>>
>> at
>> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>>
>> at
>> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>>
>> at
>> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>>
>> at
>> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>>
>> at
>> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>>
>> at
>> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>>
>> at
>> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 2:30 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
>> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Who knows about this group id? Who's suppose to set it to the topology?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru,
>>
>>
>>
>> As part of the grouping I need to add the groupId to a Member event (e.g.
>> MemberStartedEvent, MemberActivatedEvent).
>>
>> Currently I added the composite application data to the Topology by
>> generating a event and respective event processor (similar to Service)
>> which works fine for the autoscaler.
>>
>> However it seems that the CloudController uses its own registry and the
>> application info is missing when the topology is restored so it seems I
>> have to push the application data also to the cloud controller.
>>
>> The only interface available seems to be the soap api
>> (CloudControllerService.wsdl) which I would like to avoid (complicated and
>> time intensive). Is there a better way to push the data, can I use a
>> topology event, share the manager registry or do you have any other
>> recommendations ?
>>
>> In case the soap API is the only option, is there a better way to
>> generate the CloudControllerService .wsdl or do I have to manually edit in
>> my API changes ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Tuesday, May 20, 2014 9:15 AM
>> *To:* Isuru Haththotuwa
>> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Sounds good
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Tuesday, May 20, 2014 4:41 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>>
>> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Can you give me a quick json example to make sure I won’t misunderstand ?
>>
>> We would still be using the component JSON definitions as per your
>> proposal, since if we put everything inline it would be one large JSON
>> file. In the top level Composite Application Definition (which is the more
>> appropriate name for subscription Plan, as suggested by Lakmal), you can
>> refer the previously deployed groups by using the alias. So, the last
>> example that you have given in the doc would be a valid JSON sample. Hope
>> this is clear. Do let me know if its not.
>>
>>
>>
>>
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 11:33 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Lakmal,
>>
>>
>>
>> A few clarifications:
>>
>>
>>
>> I think based on the last proposal we can consolidate the definition of
>> components, dependencies in one definition and deploy all at once as
>> subscription plan, correct (simplifies the deployment) ?
>>
>>
>>
>>
>>
>> Yes, but as the first step, will have separate deployers for groups,
>> components and then composite application (which is you have mention
>> subscription plan).
>>
>>
>>
>> Also, Isuru pointed out that the term application might be confusing
>> so I suggested subscription plan (or something similar), do we still want
>> to go with it ?
>>
>>
>>
>>
>>
>> What i suggest to call it composite application.
>>
>>
>>
>> And composite application can refer, previous deployed group, components.
>> components can refer cartridge definitions. And dependancies (start order,
>> kill order) should be defined in the top level, which is in composite
>> application definition.
>>
>>
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Monday, May 19, 2014 10:58 AM
>> *To:* Martin Eppel (meppel); Lakmal Warusawithana
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Sorry, hit the send button to soon:
>>
>>
>>
>> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
>> checkin.
>>
>>
>>
>> Do we have already a planned date for 4.1 Milestone 1 ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> ...
>>
>> [Message clipped]
>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
Awesome!
On Thu, Jun 5, 2014 at 8:01 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> It worked,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 4:03 PM
>
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Thu, Jun 5, 2014 at 3:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Ok, I see – I think I don’t understand the correct process to add a new
> interface to the CloudController.
>
>
>
> Is this the correct procedure to develop the SOAP API for the could
> controller : ?
>
>
>
> 1. Implement the new API in CloudControllerServiceImpl
>
> 2. Recompile cloud controller project and patch the server
>
> 3. Retrieve the wsdl (Cloudcontroller.wsdl) and place it in my
> build environment
>
> 3.2 place it in the corresponding service stub and replace the existing
> wsdl under src/main/resurces and build the project.
>
> 4. Implement the API in CloudControllerServiceClient, recompile
> all projects and redeploy binaries
>
> Correct.
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 2:34 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> That is a risky thing to do :-) I suggest you to start the product with
> your patch and then access the auto-generated wsdl using
> https://<host>:<port>/services/CloudControllerService?wsdl
>
>
>
> On Thu, Jun 5, 2014 at 1:31 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I edited my changes manually …
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 12:16 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> How did you generate the WSDL, Martin?
>
>
>
> On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
> Hi Martin,
>
> Can you send the locally built Cloud Controller jar (with your changes),
> so that we can try to generate the WSDL.
>
>
>
> On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I patched it but still see the same issue, what’s the best way to debug
> this ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 11:20 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> I remember you faced a similar issue earlier. So, just to make sure :-)
>
>
>
> On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I recompiled all jars, generated the zips and installed the system from
> the installer zip files, you think it is still necessary ?
>
> (but I’ll try anyway - recompile service stub and patch)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 10:58 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you try to patch the CC service stub to the running product and check?
>
>
>
> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Here we go, see attached files:
>
>
>
> ServiceUtils invokes the API
>
> CloudControllerClient, CloudControllerServiceImpl for implementation
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 9:45 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> ...
>
> [Message clipped]
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
It worked,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Wednesday, June 04, 2014 4:03 PM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Thu, Jun 5, 2014 at 3:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Ok, I see – I think I don’t understand the correct process to add a new interface to the CloudController.
Is this the correct procedure to develop the SOAP API for the could controller : ?
1. Implement the new API in CloudControllerServiceImpl
2. Recompile cloud controller project and patch the server
3. Retrieve the wsdl (Cloudcontroller.wsdl) and place it in my build environment
3.2 place it in the corresponding service stub and replace the existing wsdl under src/main/resurces and build the project.
4. Implement the API in CloudControllerServiceClient, recompile all projects and redeploy binaries
Correct.
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 2:34 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
That is a risky thing to do :-) I suggest you to start the product with your patch and then access the auto-generated wsdl using https://<host>:<port>/services/CloudControllerService?wsdl<https://%3chost%3e:%3cport%3e/services/CloudControllerService?wsdl>
On Thu, Jun 5, 2014 at 1:31 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I edited my changes manually …
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 12:16 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
How did you generate the WSDL, Martin?
On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>> wrote:
Hi Martin,
Can you send the locally built Cloud Controller jar (with your changes), so that we can try to generate the WSDL.
On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I patched it but still see the same issue, what’s the best way to debug this ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 11:20 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
I remember you faced a similar issue earlier. So, just to make sure :-)
On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I recompiled all jars, generated the zips and installed the system from the installer zip files, you think it is still necessary ?
(but I’ll try anyway - recompile service stub and patch)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 10:58 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
On Thu, Jun 5, 2014 at 3:24 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Ok, I see – I think I don’t understand the correct process to add a new
> interface to the CloudController.
>
>
>
> Is this the correct procedure to develop the SOAP API for the could
> controller : ?
>
>
>
> 1. Implement the new API in CloudControllerServiceImpl
>
> 2. Recompile cloud controller project and patch the server
>
> 3. Retrieve the wsdl (Cloudcontroller.wsdl) and place it in my
> build environment
>
3.2 place it in the corresponding service stub and replace the existing
wsdl under src/main/resurces and build the project.
> 4. Implement the API in CloudControllerServiceClient, recompile
> all projects and redeploy binaries
>
Correct.
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 2:34 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> That is a risky thing to do :-) I suggest you to start the product with
> your patch and then access the auto-generated wsdl using https://
> <host>:<port>/services/CloudControllerService?wsdl
>
>
>
> On Thu, Jun 5, 2014 at 1:31 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I edited my changes manually …
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 12:16 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> How did you generate the WSDL, Martin?
>
>
>
> On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
> Hi Martin,
>
> Can you send the locally built Cloud Controller jar (with your changes),
> so that we can try to generate the WSDL.
>
>
>
> On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I patched it but still see the same issue, what’s the best way to debug
> this ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 11:20 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> I remember you faced a similar issue earlier. So, just to make sure :-)
>
>
>
> On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I recompiled all jars, generated the zips and installed the system from
> the installer zip files, you think it is still necessary ?
>
> (but I’ll try anyway - recompile service stub and patch)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 10:58 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you try to patch the CC service stub to the running product and check?
>
>
>
> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Here we go, see attached files:
>
>
>
> ServiceUtils invokes the API
>
> CloudControllerClient, CloudControllerServiceImpl for implementation
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 9:45 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Ok, I see – I think I don’t understand the correct process to add a new interface to the CloudController.
Is this the correct procedure to develop the SOAP API for the could controller : ?
1. Implement the new API in CloudControllerServiceImpl
2. Recompile cloud controller project and patch the server
3. Retrieve the wsdl (Cloudcontroller.wsdl) and place it in my build environment
4. Implement the API in CloudControllerServiceClient, recompile all projects and redeploy binaries
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Wednesday, June 04, 2014 2:34 PM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
That is a risky thing to do :-) I suggest you to start the product with your patch and then access the auto-generated wsdl using https://<host>:<port>/services/CloudControllerService?wsdl
On Thu, Jun 5, 2014 at 1:31 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I edited my changes manually …
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 12:16 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
How did you generate the WSDL, Martin?
On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>> wrote:
Hi Martin,
Can you send the locally built Cloud Controller jar (with your changes), so that we can try to generate the WSDL.
On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I patched it but still see the same issue, what’s the best way to debug this ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 11:20 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
I remember you faced a similar issue earlier. So, just to make sure :-)
On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I recompiled all jars, generated the zips and installed the system from the installer zip files, you think it is still necessary ?
(but I’ll try anyway - recompile service stub and patch)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 10:58 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
Hi Martin,
That is a risky thing to do :-) I suggest you to start the product with
your patch and then access the auto-generated wsdl using https://
<host>:<port>/services/CloudControllerService?wsdl
On Thu, Jun 5, 2014 at 1:31 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> I edited my changes manually …
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 12:16 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> How did you generate the WSDL, Martin?
>
>
>
> On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
> Hi Martin,
>
> Can you send the locally built Cloud Controller jar (with your changes),
> so that we can try to generate the WSDL.
>
>
>
> On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I patched it but still see the same issue, what’s the best way to debug
> this ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 11:20 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> I remember you faced a similar issue earlier. So, just to make sure :-)
>
>
>
> On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I recompiled all jars, generated the zips and installed the system from
> the installer zip files, you think it is still necessary ?
>
> (but I’ll try anyway - recompile service stub and patch)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 10:58 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you try to patch the CC service stub to the running product and check?
>
>
>
> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Here we go, see attached files:
>
>
>
> ServiceUtils invokes the API
>
> CloudControllerClient, CloudControllerServiceImpl for implementation
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 9:45 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
I edited my changes manually …
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Wednesday, June 04, 2014 12:16 PM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
How did you generate the WSDL, Martin?
On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>> wrote:
Hi Martin,
Can you send the locally built Cloud Controller jar (with your changes), so that we can try to generate the WSDL.
On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I patched it but still see the same issue, what’s the best way to debug this ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 11:20 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
I remember you faced a similar issue earlier. So, just to make sure :-)
On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I recompiled all jars, generated the zips and installed the system from the installer zip files, you think it is still necessary ?
(but I’ll try anyway - recompile service stub and patch)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 10:58 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
How did you generate the WSDL, Martin?
On Thu, Jun 5, 2014 at 12:35 AM, Isuru Haththotuwa <is...@apache.org>
wrote:
> Hi Martin,
>
> Can you send the locally built Cloud Controller jar (with your changes),
> so that we can try to generate the WSDL.
>
>
> On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
>> I patched it but still see the same issue, what’s the best way to debug
>> this ?
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 11:20 AM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> I remember you faced a similar issue earlier. So, just to make sure :-)
>>
>>
>>
>> On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> I recompiled all jars, generated the zips and installed the system from
>> the installer zip files, you think it is still necessary ?
>>
>> (but I’ll try anyway - recompile service stub and patch)
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 10:58 AM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Can you try to patch the CC service stub to the running product and check?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Here we go, see attached files:
>>
>>
>>
>> ServiceUtils invokes the API
>>
>> CloudControllerClient, CloudControllerServiceImpl for implementation
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Wednesday, June 04, 2014 9:45 AM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
>> wrote:
>>
>> I am sorry Martin, but it'll easy for me, if you can attach the code diff
>> of CC component.
>>
>> Thanks.
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Attached to the email,
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 10:20 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Can you please send a diff over?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> No, but I modeled / followed the deployCartridgeDefinition API which I
>> think passes a bean as parameter ?
>>
>>
>>
>> Is it required to define a bean ? Where and how ?
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 8:57 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Did you introduce a new bean class?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Actually,
>>
>>
>>
>> Below are all the changes I made to the wsdl for the new API, all I want
>> at this point is to pass a string parameter in the API:
>>
>>
>>
>> <wsdl:operation name="deployCompositeApplicationDefinition">
>>
>> <soap:operation
>> soapAction="urn:deployCompositeApplicationDefinition"
>> style="document"></soap:operation>
>>
>> <wsdl:input>
>>
>> <soap:body use="literal"></soap:body>
>>
>> </wsdl:input>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <soap:fault use="literal"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>>
>> </wsdl:fault>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidIaasProviderException">
>>
>> <soap:fault use="literal"
>> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>>
>> </wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>> <wsdl:operation name="deployCompositeApplicationDefinition">
>>
>> <soap12:operation
>> soapAction="urn:deployCompositeApplicationDefinition"
>> style="document"></soap12:operation>
>>
>> <wsdl:input>
>>
>> <soap12:body use="literal"></soap12:body>
>>
>> </wsdl:input>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <soap12:fault use="literal"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>>
>> </wsdl:fault>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidIaasProviderException">
>>
>> <soap12:fault use="literal"
>> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>>
>> </wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>> <wsdl:operation
>> name="deployCompositeApplicationDefinition">
>>
>> <http:operation
>> location="deployCompositeApplicationDefinition"></http:operation>
>>
>> <wsdl:input>
>>
>> <mime:content type="text/xml"
>> part="parameters"></mime:content>
>>
>> </wsdl:input>
>>
>> </wsdl:operation>
>>
>>
>>
>>
>>
>> <xs:element
>> name="deployCompositeApplicationDefinition">
>>
>> <xs:complexType>
>>
>> <xs:sequence>
>>
>> <xs:element minOccurs="0"
>> name="compositeApplicationConfig" nillable="true"
>> type="xs:string"></xs:element>
>>
>> </xs:sequence>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>>
>>
>>
>>
>>
>>
>> <wsdl:message
>> name="deployCompositeApplicationDefinitionRequest">
>>
>> <wsdl:part name="parameters"
>> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>>
>> </wsdl:message>
>>
>>
>>
>> <wsdl:operation
>> name="deployCompositeApplicationDefinition">
>>
>> <wsdl:input
>> message="ns:deployCompositeApplicationDefinitionRequest"
>> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>>
>> <wsdl:fault
>> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
>> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>>
>> <wsdl:fault
>> message="ns:CloudControllerServiceInvalidIaasProviderException"
>> name="CloudControllerServiceInvalidIaasProviderException"
>> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>>
>>
>> <xs:element
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <xs:complexType>
>>
>> <xs:sequence>
>>
>> <xs:element minOccurs="0"
>> name="InvalidCompositeApplicationDefinitionException" nillable="true"
>> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>>
>> </xs:sequence>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>>
>>
>> <wsdl:message
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <wsdl:part name="parameters"
>> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>>
>> </wsdl:message>
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Tuesday, June 03, 2014 8:47 PM
>> *To:* dev@stratos.apache.org
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Yes,
>>
>>
>>
>> I did add the new api in the cloudcontroller.wsdl and regenerated the
>> stub (recompiled successfully all projects, including service-stub, etc …).
>> The error happens during run time when I invoke the API, are there any
>> debug traces I can turn on for axis which might help ?
>>
>>
>>
>>
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Tuesday, June 03, 2014 8:11 PM
>> *To:* dev@stratos.apache.org
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
>> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Marin,
>>
>> If you have changed the CloudController interface, you will need to use
>> the new cloud controller WSDL and re-generate the Cloud Controller service
>> stub code. This error can be a result of this. Let me know if you have not
>> done this so that I can give some guide lines.
>>
>>
>>
>> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> When I invoke my soap API I get the following exception, what ‘s the best
>> way to troublshoot / debug this ? What could be the issue ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
>> occurred while trying to invoke service method
>> deployCompositeApplicationDefinition
>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>>
>> java.lang.ArrayIndexOutOfBoundsException: 1
>>
>> at
>> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>>
>> at
>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>>
>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>
>> at
>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>>
>> at
>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>>
>> at
>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>
>> at
>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>
>> at
>> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>
>> at
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>
>> at
>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>
>> at
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>
>> at
>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>
>> at
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>
>> at
>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
>> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
>> occurred while trying to invoke service method
>> deployCompositeApplicationDefinition
>> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>>
>> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
>> service method deployCompositeApplicationDefinition
>>
>> at
>> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>>
>> at
>> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>>
>> at
>> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>>
>> at
>> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>>
>> at
>> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>>
>> at
>> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>>
>> at
>> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 2:30 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
>> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Who knows about this group id? Who's suppose to set it to the topology?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru,
>>
>>
>>
>> As part of the grouping I need to add the groupId to a Member event (e.g.
>> MemberStartedEvent, MemberActivatedEvent).
>>
>> Currently I added the composite application data to the Topology by
>> generating a event and respective event processor (similar to Service)
>> which works fine for the autoscaler.
>>
>> However it seems that the CloudController uses its own registry and the
>> application info is missing when the topology is restored so it seems I
>> have to push the application data also to the cloud controller.
>>
>> The only interface available seems to be the soap api
>> (CloudControllerService.wsdl) which I would like to avoid (complicated and
>> time intensive). Is there a better way to push the data, can I use a
>> topology event, share the manager registry or do you have any other
>> recommendations ?
>>
>> In case the soap API is the only option, is there a better way to
>> generate the CloudControllerService .wsdl or do I have to manually edit in
>> my API changes ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Tuesday, May 20, 2014 9:15 AM
>> *To:* Isuru Haththotuwa
>> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Sounds good
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Tuesday, May 20, 2014 4:41 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>>
>> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Can you give me a quick json example to make sure I won’t misunderstand ?
>>
>> We would still be using the component JSON definitions as per your
>> proposal, since if we put everything inline it would be one large JSON
>> file. In the top level Composite Application Definition (which is the more
>> appropriate name for subscription Plan, as suggested by Lakmal), you can
>> refer the previously deployed groups by using the alias. So, the last
>> example that you have given in the doc would be a valid JSON sample. Hope
>> this is clear. Do let me know if its not.
>>
>>
>>
>>
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 11:33 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Lakmal,
>>
>>
>>
>> A few clarifications:
>>
>>
>>
>> I think based on the last proposal we can consolidate the definition of
>> components, dependencies in one definition and deploy all at once as
>> subscription plan, correct (simplifies the deployment) ?
>>
>>
>>
>>
>>
>> Yes, but as the first step, will have separate deployers for groups,
>> components and then composite application (which is you have mention
>> subscription plan).
>>
>>
>>
>> Also, Isuru pointed out that the term application might be confusing
>> so I suggested subscription plan (or something similar), do we still want
>> to go with it ?
>>
>>
>>
>>
>>
>> What i suggest to call it composite application.
>>
>>
>>
>> And composite application can refer, previous deployed group, components.
>> components can refer cartridge definitions. And dependancies (start order,
>> kill order) should be defined in the top level, which is in composite
>> application definition.
>>
>>
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Monday, May 19, 2014 10:58 AM
>> *To:* Martin Eppel (meppel); Lakmal Warusawithana
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Sorry, hit the send button to soon:
>>
>>
>>
>> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
>> checkin.
>>
>>
>>
>> Do we have already a planned date for 4.1 Milestone 1 ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Monday, May 19, 2014 10:52 AM
>> *To:* 'Lakmal Warusawithana'
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 9:45 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Lakmal,
>>
>>
>>
>> This sounds good to me, how do you suggest to coordinate the development.
>> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>>
>>
>>
>>
>>
>> No, lets work on master branch. Which part do you like to work on? Can
>> you start work on deploy these new definitions?
>>
>>
>>
>> Need to have persist model also. IsuruH, can you guys come up with it.
>>
>>
>>
>> Melan, can you start extent Composite Application create event?
>>
>>
>>
>> Lahiru can you start working on Composite Application monitor?
>>
>>
>>
>> Will target 4.1.0 Milestone 1.
>>
>>
>>
>> Is this look like a plan :)
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 2:06 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Shall we start implementation, I can see following tasks.
>>
>> - Extending SM API for
>>
>>
>> - deploy components
>> - deploy group
>> - deploy composite application definition
>>
>>
>> - persist above deployment
>> - validation
>> - Extend SM for Composite Application creation in the topology
>> (earlier we had cluster create)
>> - Extend Auto Scaler (implementing Composite Application monitor)
>>
>>
>>
>> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
>> wrote:
>>
>> Thanks Martin, get clear idea now.
>>
>>
>>
>> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, Melan
>>
>>
>>
>> Based on our last night conversation I want to slightly modify the
>> previous grouping proposal:
>>
>>
>>
>> Instead of defining an application (misleading naming) we define a
>> subscription plan:
>>
>>
>>
>> The subscription defines the following:
>>
>> + grouping of components (or subscription_groups)
>>
>> + subscription_groups logically group subscribed cartridges and other
>> subscription groups (nested grouping) ( by means of referencing their alias)
>>
>> + dependencies between components (startup, termination, etc)
>>
>> + inline definition of subscription_groups in json doc (see example
>> further down)
>>
>>
>>
>> Going with this proposal I think we could leave current cartridge
>> subscription unchanged but tie the subscribed cartridges to a subscription
>> plan. Startup and termination dependencies will be checked by the
>> autoscaler.
>>
>>
>>
>> Please see also the section “*Subscription Plan”*in the shared google doc
>>
>>
>>
>>
>>
>> I would like to call it as "Composite Application" instead of
>> "Subscription Plan".
>>
>>
>>
>>
>>
>> What do you think ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>>
>>
>>
>>
>>
>> *Json example:*
>>
>>
>>
>> {
>>
>> "subscriptionPlanId": "subscriptionId",
>>
>> "alias": "subscription name",
>>
>> "components": [
>>
>> {
>>
>> "configGroup": {
>>
>> "alias": "group1",
>>
>> "subscribables": [
>>
>> "c1",
>>
>> "group2"
>>
>> ],
>>
>> "dependencies": {
>>
>> "startup_order": [
>>
>> {
>>
>> "key": "c1",
>>
>> "value": "c2"
>>
>> }
>>
>> ],
>>
>> "kill_behavior": "asymmetrical"
>>
>> }
>>
>> }
>>
>> },
>>
>> {
>>
>> "configGroup": {
>>
>> "alias": "group2",
>>
>> "subscribables": [
>>
>> "c3"
>>
>> ]
>>
>> }
>>
>> }
>>
>> ]
>>
>> }
>>
>>
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Saturday, May 17, 2014 2:44 PM
>> *To:* Isuru Haththotuwa
>>
>>
>> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
>> Warusawithana; Shaheedur Haque (shahhaqu)
>>
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Absolutely,
>>
>>
>>
>> We want to incorporate the feedback of all the experts but in the
>> meantime we can get started based on what we know and then gradually
>> enhance.
>>
>>
>>
>> IMHO it would also make sense to start developing the feature on a
>> separate branch (of RC3 tag ) and gradually improve the feature until we
>> feel it is ready to be incorporated into the mainline, this would allow us
>> to have something ready by the end of the month without destabilizing the
>> master branch ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Saturday, May 17, 2014 10:16 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
>> Warusawithana; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Thanks a lot for your efforts. Will try to have a look at the code before
>> the call.
>>
>> IMHO, before adding our changes, it is better to share the intended
>> design with the dev list and get feedback as well. We might need to come
>> with some sequence diagrams as well. I was planning to do that after we
>> have the call tomorrow. This is a crucial and a core change, plus a very
>> useful feature to Stratos, so better to get feedback from the experts in
>> the dev list. WDYT?
>>
>> I prepared some JSON definition referring your ones, in which I have only
>> one single JSON definition for the group subscription. My intention is to
>> re-use the Subscription model that we have in Stratos Manager and extend it
>> to support groups and dependencies. Lets discuss during the call, and
>> decide how to proceed.
>>
>>
>>
>> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, Melan
>>
>>
>>
>> Attached is the code I started implementing:
>>
>> 1. Added REST API for application deployment
>>
>> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
>> infrastructure)
>>
>>
>>
>> I also attached the sample JSON I used to test the REST API:
>>
>>
>>
>> Some of the code is experimental, e.g publishing the
>> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
>> messaging infrastructure and to be able to implement and test the backend
>> changes (application json) in the autoscaler (not there yet).
>>
>>
>>
>> I think there is still a bunch of things missing like an application
>> deployment / subscriptions manager (similar to deploying a cartridge, etc
>> …), but I am not entirely sure what you guys have in mind there.
>>
>> I do have some idea what should be done in the autoscaler to add
>> dependencies check, so I was thinking you could focus on the deployment
>> logic, messaging, etc … while I take a look at the autoscaler stuff.
>>
>> Anyway, this is just a suggestion, take a look and we can discuss more
>> details in our call.
>>
>>
>>
>> Regards
>>
>>
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Friday, May 16, 2014 8:16 PM
>> *To:* Isuru Haththotuwa
>>
>>
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
>> Haque (shahhaqu); Melan Jayasingha
>>
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Isuru,
>>
>>
>>
>> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
>> time would be Sunday 8.30 AM IST)
>>
>>
>>
>> Regards
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
>> *Sent:* Friday, May 16, 2014 10:50 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
>> Haque (shahhaqu); Melan Jayasingha
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru,
>>
>>
>>
>> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
>> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>>
>> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
>> IST)
>>
>>
>>
>> How will we connect, I haven’t used Google hangout yet ?
>>
>> We can connect through either Google hangout/ Skype, whatever easier
>> for you.
>>
>>
>>
>> I noticed that emails are getting very slowly processed on dev@stratos
>> list so I am sending it also directly to you as well,
>>
>> I too noticed that your last mail got delayed by approx. one day.
>>
>>
>>
>> Btw, we are trying to get this feature ready (at least in some basic
>> form) by the end of the month
>>
>> I'm positive we can get an initial version working by the end of this
>> month, if all goes well.
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Thursday, May 15, 2014 9:10 PM
>> *To:* dev@stratos.incubator.apache.org
>> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>>
>>
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> I'm in IST. Please suggest a convenient time for you. I would like to
>> clarify several things on the proposal and the concepts you brought in,
>> simplify it further if possible, and decide how to proceed.
>>
>>
>>
>> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, see inline
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Wednesday, May 14, 2014 1:12 PM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
>> Haque (shahhaqu)
>>
>>
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>>
>> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, Shaheed
>>
>>
>>
>>
>>
>> Ad “kill behaviour”:
>> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”,
>> “kill-none” (btw, was left out accidentally, thanks for pointing it out),
>>
>>
>>
>> Ad single definition:
>>
>>
>>
>> Makes sense too, however we should still consider that the json
>> definition can become fairly complex when multiple components and sub
>> components are defined, that’s the reason I think it would still make sense
>> to keep it modular, at least as an option.
>>
>> +1
>>
>> Martin: +1
>>
>> (btw, I assume you suggest instead of defining each component in
>> separate files we aggregate them in the same application json document,
>> but not like a “in-line” definition of each component?)
>>
>> Exactly . This is fine as the first step IMHO.
>>
>>
>>
>> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>>
>> Does this include other definitions like cartridge info definition,
>> deployment policy, autoscaling definition, etc ... ?
>>
>> No. I agree that if we do this, it will be an unmanageably large JSON
>> definition. We can refer to the relevant policies (which are already
>> deployed) in the service group definition. The service group should contain
>> the group specific details IMHO, as the dependencies, startup orders, kill
>> behaviours, etc. WDYT?
>>
>>
>>
>> Martin: +1 (see my example json definitions)
>>
>>
>>
>> We could provide eventually both options, in-line with all components
>> defined in one document (simple configurations) or modular, where
>> components are deployed (un-deployed) individually (for large, complex
>> configurations).
>>
>>
>>
>> In the shared document I provided a full example of the json definition
>> of an application, and just adding a few components seemed to make the
>> document fairly large or complex (see chapter “Json example”).
>>
>> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
>> we can decide on the next steps.
>>
>>
>>
>> Martin: sounds good, Friday would be Thursday my time, which time zone
>> are you in ?
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>>
>>
>>
>>
>>
>> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
>> *Sent:* Tuesday, May 13, 2014 6:08 AM
>> *To:* dev@stratos.incubator.apache.org
>> *Cc:* Lakmal Warusawithana
>>
>>
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin, Shaheedur and all,
>>
>> Went through the shared document for the proposal. good work! One small
>> suggestion, additionally to deploying component JSON definitions one by
>> one, shall we allow to deploy a one single definition (that will have all
>> the dependencies and other information defined) to be deployed? WDYT? IMHO
>> this will be more user friendly.
>>
>>
>>
>>
>>
>> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
>> shahhaqu@cisco.com> wrote:
>>
>> A couple of points on “kill behaviour”:
>>
>>
>>
>> · I don’t think the terms “asymmetric” and “symmetric” capture
>> the behaviours I had in mind. For me there were two use cases:
>>
>> o Assuming A depends on B and B depends on C
>>
>> o And B fails
>>
>> o Use case #1 is that only A should be killed and allowed to restart.
>> We could call this kill behaviour “kill-dependents” since A is dependent on
>> B. This fits the standard n-tier architecture where A is the front end, B
>> is the application logic and C is the database.
>>
>> o Use case #2 is that both A and C should be killed and allowed to
>> restart. We could call this “kill-all”. This fits the use case of a poorly
>> behaved application where everything needs a restart on any failure.
>>
>> · Did we accidentally drop the third value, i.e. “kill-none”, or
>> did we somehow convince ourselves this was not needed? I’m actually tending
>> towards the latter since if kill-none were a usable value, it seems to
>> contradict that there was a dependency in the first place.
>>
>> +1.
>>
>>
>>
>> Thanks, Shaheed
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* 07 May 2014 06:48
>>
>>
>> *To:* dev@stratos.incubator.apache.org
>> *Cc:* Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Lakmal,
>>
>> See inline "Martin:"
>>
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>>
>> Sent: Tuesday, May 06, 2014 3:25 AM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Hi Martin,
>>
>> Thanks for adding content, I had offline discussion with Imesh, Lahiru
>> and IsuruH on this, we can start implementing step by step. will do this by
>> milestone by milestone approach.
>>
>> Further I think we can used same topology topic to create this model.
>> With this root node of the topology will be application. From the auto
>> scaler we can implement application monitor to handle this. service
>> clusters will be sub set of application.
>>
>> Martin: +1, dependencies have to be included
>>
>>
>> As the first step we reused (partitions, auto scaling policy, deployment
>> policy, cartridge definitions) existing definitions to define application
>> with new requirement like grouping, dependency, start order ..etc. Shall we
>> create sample json ( with sample auto scaling policies, deployment
>> policies, cartridges ..etc) to get some idea for the complexity? (still I
>> am figuring out some definition like subscribable and some real sample json)
>>
>> Martin: +1, Actually, the json example I posted on Google docs assumes
>> that we'll be reusing all the existing artifacts (like cartridge definition
>> etc ...). Cartridges, policies, componenents are referenced by name so each
>> of these have to be deployed separately (as they are today) and when the
>> application is deployed it will reference the deployed artifacts - what do
>> you think ?
>> We'll might have to add some extra properties - I'll work on a complete
>> example and post the progress on Google Doc.
>>
>>
>>
>> Then we should divide each work item and start implementing. Ideally in
>> the milestone one should not break existing, but reused them. Then later we
>> can identifying what need to improve/change existing components.
>>
>> Martin: +1, would it speed up / simply implementation if we start working
>> on a branch ? Also, I totally agree, all code change should be
>> complementary without breaking any existing features
>>
>>
>> Later this week we can have google hangout session to discuss further on
>> this.
>>
>> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we
>> can do a hangout either today, tomorrow or then again next week Wednesday.
>>
>>
>>
>> Seems like we have to wait till next Friday, next week, Wednesday and
>> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
>> identify initial task and will post here.
>>
>>
>> Lets have a hangout at a convenient time to discuss the next steps.
>>
>>
>> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Works for me, I’ll add the content
>>
>> Thanks
>>
>> Martin
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Sunday, May 04, 2014 8:56 PM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Hi Martin,
>>
>> Thanks for detail description. We can create wiki space , but IMHO its
>> easy if we can use public google doc. I have created [1] shall we add our
>> proposal into it. May be we need some hangout session to discuss further on
>> this.
>>
>> [1]
>> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>>
>>
>> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi Lakmal,
>>
>> I summarized and formatted the discussion points from the previous
>> email(s) thread for better readability and reference.
>> Btw, what is the appropriate way going forward to elevate the discussion
>> to a feature request (JIRA) ? Also, how do we typically discuss feature
>> requests like this, the email format makes it somewhat difficult to read
>> and it is easy to overlook posted discussion points ? Is there a wiki to
>> post and discuss proposals like this ?
>>
>> Thanks
>>
>> Martin
>>
>> Grouping of Cartridges / Services:
>>
>> 1. Application model
>> 1.1 General model
>> See proposal below
>> 1.2 Json
>> see proposal below
>>
>> 2. Application deployment
>> 2.1 Deployment
>> + application deployment (deploy)
>> + application removal (undeploy)
>> 2.2 Subscription
>> + subscription (subscribe)
>> + cancelation of subscription (unsubscribe)
>>
>> 2. Distributing Application deployment
>> 2.1 Publishing application deployment
>> + new topic to publish application events, map, status
>> 2.2 Subscribing to published application deployment
>> + Autoscaler
>>
>> 3. Dependency calculation
>> 3.1 Building static dependency tree
>> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
>> + Dependency tree determines all dependencies,
>> + symmetrical / asymmetrical dependencies
>> + Dependency checks member status for all dependencies
>> successful for at least one member in ACTIVE state, failed
>> for no member in ACTIVE state
>> 3.1.1.1 Enhancing autoscaler rules
>> + Integrate with
>> mincheck.drl to control instance creation,
>> termination rule to control instance termination
>>
>>
>> 4. Application event model
>> + provides state information for application, application components
>> like
>> Up, Down, partial up
>>
>>
>>
>> From: Martin Eppel (meppel)
>> Sent: Thursday, May 01, 2014 3:26 PM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: RE: [Discuss] Grouping of services (cartridges)
>>
>> Hi Lakmal at al
>>
>> Here is a first draft for an application model as a starting point, it
>> builds on the previous discussions (and probably leaves out a few things):
>> Also, to add to Shaheeds point there is probably an event model around
>> this as well.
>>
>> Definitions:
>> + composite structure
>> + A component is sectioned into description, components, configuration,
>> dependencies.
>> + Components are typed (application, subscribables, scalable)
>> + Application is also component
>>
>> + description ... properties (like name, type)
>> + components ... children, nested
>> + configuration ... properties
>> + dependencies ... puts immediate components (children) in relation
>>
>> Generic Component format:
>> + description (component specific properties)
>> + components (by reference)
>> + configuration
>> - described through properties
>> + properties can be grouped, group can be referenced
>> + dependencies
>> - described through properties
>>
>>
>> In json:
>>
>> Generic Component:
>>
>> Component:
>>
>> {
>> "description": [
>> ],
>> "configuration":[
>> ],
>> "components":[
>> ],
>> "dependencies":[
>> ]
>> }
>>
>>
>> Application:
>>
>> {
>> "description": [
>> {"name":"application name"},
>> {"types":["application", "subscribable"]}
>> ],
>> "configuration":[
>> {"deployment":"deployment_policy_name"}
>> ],
>> "components":[
>> {"subscribables": ["subscribable name",
>> "subscribable name",
>> "subscribable name"]}
>> ],
>> "dependencies":[
>> {"startup_order": [
>> {"dependant name": "subscribable name"},
>> {"dependant name": "subscribable name"}
>> ]},
>> {"kill_behavior": "symmetrical or asymmetrical"}
>> ]
>> }
>>
>> Subscribable (aka groups):
>>
>> {
>> "description": [
>> {"name":"component name"},
>> {"types":["subscribable"]}
>> ],
>> "configuration":[
>> ],
>> "components":[
>> {"subscribables": ["subscribable name",
>> "subscribable name",
>> "subscribable name"]}
>> ],
>> "dependencies":[
>> {"startup_order": [
>> {"dependant name": "subscribable name"},
>> {"dependant name": "subscribable name"}
>> ]},
>> {"kill_behavior": "symmetrical / asymmetrical"}
>> ]
>> }
>>
>> Cartridge Component:
>> {
>> "description": [
>> {"name":"component name"},
>> {"types":["subscribable", "scalable"]}
>> ],
>> "configuration":[
>> {"scaling":"autoscaling policy name"},
>> {"info":"cartridge info"}
>> ],
>> "components":[
>> ],
>> "dependencies":[
>> ]
>> }
>>
>>
>> Thanks
>>
>> Martin
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Thursday, May 01, 2014 1:18 AM
>> To: Shaheedur Haque (shahhaqu)
>> Cc: dev@stratos.incubator.apache.org
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
>> Hi,
>>
>> Sorry for top-posting, but one additional thing that I feel should be
>> considered is a minimal set of lifecycle events for the group; for example:
>>
>> - "all members of a group are now active", aka "group active".
>>
>> - "at least one member of an existing group has failed", aka "group down".
>>
>> This is clearly an add-on to the core functionality but is needed to
>> round the feature out.
>>
>>
>> +1
>>
>> Thanks, Shaheed
>>
>>
>> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
>> Hi Martin,
>>
>> With the current implementation I think and agree its better to go with
>> json as defining configuration data. Summarizing proposal and the
>> discussion I can think following data model. Here I am thinking
>> implementing single configuration data (application deployment definition)
>> model, which can define full configuration of an composite application. we
>> should take it down to several milestones. we can start on enhancing
>> backend services in initial milestone and later will come with aggregating
>> them. All of your thoughts welcome.
>>
>> Can you come up with some definition format. (may be json format). Also
>> see my inline comment.
>>
>> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi Lakmal,
>>
>> Below are ideas on how to break up the feature into a high level ToDo
>> list (with some suggestions and questions) :
>>
>> + Defining configuration data format – for example json or chef / recipe
>> ++ would using recipes affect all configuration data or is it
>> restricted to the grouping / dependency configuration only ?
>>
>> IMHO, its may easy with json, considering current existing services.
>>
>>
>> + Adding the new model to stratos manager and integrate with existing
>> model:
>> ++ Group model (nested groups, group properties)
>> ++ Dependency model
>> ++ in the current model we have a Subscriptions and Deployment
>> manager, I think we need something similar to handle the groups which in
>> turn invokes deployment / subscriptions manager
>>
>>
>> +1
>>
>> + Defining runtime dependency conditions:
>> ++ Calculating the dependency for a cartridge (service) dynamically
>> by walking the dependency tree and checking the state of dependencies (e.g.
>> member state)
>>
>>
>> +1
>>
>> + Integrate the dynamic aspect of the dependency model (checking the
>> conditions of dependencies and act accordingly):
>> ++ As of now I think it has to be integrated with the autoscaler as
>> it the entity which controls instances (members) and checks their states
>>
>> Yes, we need to enhance auto scaler
>>
>> ++ How is dependency retrieved from configuration data (e.g.
>> published through topology, beans, etc ….) ?
>>
>> We need to think on more this. IMO, we may need another topic
>> (application deployment definition) to retrieved the config data. SM can
>> define this based on provide definition, auto scaler and who ever need can
>> get that. Topology is maintaining current running state (currently its for
>> a service, we need to enhance it to application), so we can used
>> "application deployment definition" data to check with current state.
>>
>> ++ Integrate dependency checks into the runtime portion of the
>> autoscaler, e.g. add the checks to the monitor functionality of a
>> ClusterMonitor with respective actions (spawning / terminating instances)
>>
>>
>> I think we can coupled auto scaling policy with individual
>> cartridges/services.
>>
>> What do you think ?
>>
>>
>> We may need to think about deployment policy, I think with the
>> application deployment definition (with some coupling ,like some cartridges
>> may need to spin up same partition..etc). May be we are redefining
>> deployment policy with application deployment definition. (we may obsolete
>> deployment policy)
>>
>> Thanks
>>
>> Martin
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Wednesday, April 30, 2014 4:30 AM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Hi Martin,
>>
>> These use cases are very valid, and we should integrate them into 4.1.0.
>> Will go through in detail and see how we can incorporate into Stratos.
>> I have one question, why do we need hierarchical grouping, (I mean group
>> inside a group) and use case? Can't we have flat?
>>
>> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi Lakmal
>>
>> Below is a further enhanced proposal to add grouping to apache stratos:
>>
>> In a nutshell:
>>
>> By grouping cartridges together we can define characteristics which apply
>> to all cartridges alike most importantly it will allow us also to define
>> dependencies between cartridges. Groups should also be flexible enough to
>> not only contain single cartridges but also groups, allowing a hierarchical
>> structure.
>>
>> Dependencies are defined at the group level, describing the dependency
>> relationship of immediate children.
>> Other group specific properties could be defined which will apply to all
>> immediate group members.
>>
>> The subscription model will be extended to subscribe to a group in
>> addition to a cartridge (either or). To generalize we added the concept of
>> a Subscribable which can be either a group or cartridge.
>>
>> Since a Subscribale (or more specifically a group) doesn’t have scaling
>> characteristics, it seemed appropriate to add the concept of Scalable,
>> describing the scalable nature of a cartridge.
>>
>> Below is a more detailed description and a corresponding diagram:
>>
>> “Class diagram” of the logical model showing the existing description
>> files in the Diadem model, and the proposed changes.
>>
>> Description: see attachment Slide1.png
>>
>> Some notes on the new dependency system:
>>
>> * A Group can be subscribed or a Scalable can be subscribed, whereas
>> today, a Cartridge is subscribed.
>> o The new Scalable entity is needed because currently
>> Subscriptions own autoscale policies, but that doesn’t work when
>> a Group contains more than one Cartridge, because we want one
>> autoscale policy per Cartridge. However we can’t couple the
>> autoscale policy to the Cartridge because it may be re-used in
>> other Subscriptions where a different autoscale policy is desired
>> * children is the set of Subscribables in the Group.
>> o This supports hierarchical Groups.
>> * collocate says “these Children must be physically next to each other”
>> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
>> but B doesn’t point to A.
>> o These pairs define a DAG, which we use to perform a topological
>> sort of the children and get a /partial/ ordering.
>> o Any member of “children” not in a pair is a singleton started in
>> parallel to everything else.
>> * The killBehaviour says that when node X dies, we must kill:
>> o Everything else (for applications where the loss of any member
>> of “children” cannot be recovered with less impact
>> o Nothing else (for applications where any element can be restarted)
>> o Or just the things “upstream” in the startup order (for
>> conventional tiered applications)
>>
>> We think this describes all the use-cases we’ll ever see with the
>> exception of things like “n instances of A depends on B”; that can be
>> considered in the future as an enhancement.
>>
>> Startup use cases considered
>>
>> * All start in parallel: don’t specify any pairs => everything is
>> equivalent => all start in parallel
>> * All start one-after-another (total order): specify a set of pairs
>> such that there is a single contiguous list covering all the nodes,
>> e.g. A -> B -> C -> D -> E
>> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
>> -> Database], [Logging]]
>>
>> Kill use cases considered
>>
>> * One dies => all die: set killBehaviour to KillAll
>> * One dies => nothing happens: set killBehaviour to killNone
>> * One dies => its dependancies die (aka “restart from here”): specify
>> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
>> -> Database], [Logging]], Database dies => MiddleTier is restarted,
>> Logging untouched.
>>
>> Example instantiation
>>
>> Description: see attachment Slide2.png
>>
>> Thanks
>>
>> Martin
>>
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Sunday, March 30, 2014 9:09 PM
>>
>> To: dev@stratos.incubator.apache.org
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> I totally agree, I am also not in favor of complicating existing
>> components (e.g. autoscaler).
>>
>> However, I am not sure what the alternative might be to solve the
>> requirements (e.g. item 1) mentioned below.
>> The suggestion I made to enhance the autoscaler / rules are based on my
>> understanding that the autoscaler already handles similar actions (e.g. VM
>> startup, scaling and termination).
>> It seems to me a logical extension to add additional knowledge to handle
>> the boot sequencing, dependent scaling and dependent termination to the
>> autoscaler defined by optional properties in the autoscaler policy.
>>
>> From my current point of view, CAMP seems a fairly complex specification
>> which might require quite some changes to adopt.
>>
>> Sorry I could not go through CAMP in detail. Should spend sometime.
>>
>>
>> I do agree that alternatives might exist and should be discussed !?
>> Alternatives ?
>>
>> +1 for alternatives, we should not take CAMP as it is, we should see how
>> we can match with existing Stratos workflow IMO.
>>
>>
>> Thanks
>>
>> Martin
>>
>> From: damitha kumarage [mailto:damitha23@gmail.com]
>> Sent: Sunday, March 30, 2014 7:59 AM
>> To: dev@stratos.incubator.apache.org
>>
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Please see my inline comment,
>>
>> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
>> wrote:
>> Hi Martin,
>> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi,
>>
>> I think this property will be a quite useful piece to solve the grouping
>> issue:
>>
>> I also would like to suggest to add the serviceGroup to the topology map
>> (in case it is not yet available in the topology map). This will help to
>> tie together cartridges (or services) in the autoscaler and , for example
>> enable synchronized auto scaling behavior of services within a service
>> group, like synchronized scaling, sequenced boot up, etc ….
>>
>> In addition the autoscaler should be enhanced to add additional (but
>> optional properties) in the auto scaling policy related to a service group
>> to govern the respective auto scaling behavior.
>>
>> For example, related properties should identify a service group and other
>> related properties to define dependencies between the various cartridges in
>> the service group like boot sequence, scale up / down ratios, termination
>> dependencies, etc … . The property set (or json structure ) should be
>> fairly flexible as we are just about to explore this new feature and should
>> be easily expandable.
>>
>> I would think that these additions will also prove useful to integrate in
>> the long term with CAMP (or other spec) but will help to solve more
>> immediate requirements
>>
>>
>> Yes, these are very valid points in coming up with a proper grouping
>> architecture for services. Thank you for bringing them up.
>>
>> As I understand, what Sajith has done here is enabling static grouping of
>> services, by using a property for that in the cartridge deployment time.
>> What we are trying to achieve in long term is dynamic grouping of services,
>> so that we can group any available service at runtime seamlessly, according
>> to CAMP specification (or some other suitable way).
>>
>> I see three things here
>> 1) Grouping and resolve inter-dependancies between cartridges.
>> 2) Composite Artifact deployment (May be if required, according to the
>> above grouping and inter-dependancy( of cartridges. There can also be
>> intra-dependancies inside a cartridge in case of artifact deployment)
>> 3) Improved nonitoring and health check of above intricacies.
>> Instead of adopt CAMP to solve above, I think it is better to discuss and
>> find ways that fits most naturally to the existing Stratos
>> architecture(Unless CAMP is widely adopted and we are compelled to
>> adhere). Is there a way we can solve this without doing major changes to
>> existing components? For example without complicating autoscaler
>> policies/logic as suggested by Martin above?
>> Damitha
>>
>>
>>
>>
>> --
>> __________________________________________________________________
>> Damitha Kumarage
>> http://people.apache.org/
>> __________________________________________________________________
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Thanks and Regards,
>>
>> Isuru H.
>>
>> +94 716 358 048
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Thanks and Regards,
>>
>> Isuru H.
>>
>> +94 716 358 048
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Isuru Haththotuwa <is...@apache.org>.
Hi Martin,
Can you send the locally built Cloud Controller jar (with your changes), so
that we can try to generate the WSDL.
On Thu, Jun 5, 2014 at 12:24 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> I patched it but still see the same issue, what’s the best way to debug
> this ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 11:20 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> I remember you faced a similar issue earlier. So, just to make sure :-)
>
>
>
> On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> I recompiled all jars, generated the zips and installed the system from
> the installer zip files, you think it is still necessary ?
>
> (but I’ll try anyway - recompile service stub and patch)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 10:58 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you try to patch the CC service stub to the running product and check?
>
>
>
> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Here we go, see attached files:
>
>
>
> ServiceUtils invokes the API
>
> CloudControllerClient, CloudControllerServiceImpl for implementation
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 9:45 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
I patched it but still see the same issue, what’s the best way to debug this ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Wednesday, June 04, 2014 11:20 AM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
I remember you faced a similar issue earlier. So, just to make sure :-)
On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I recompiled all jars, generated the zips and installed the system from the installer zip files, you think it is still necessary ?
(but I’ll try anyway - recompile service stub and patch)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 10:58 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
I remember you faced a similar issue earlier. So, just to make sure :-)
On Wed, Jun 4, 2014 at 11:41 PM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> I recompiled all jars, generated the zips and installed the system from
> the installer zip files, you think it is still necessary ?
>
> (but I’ll try anyway - recompile service stub and patch)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 10:58 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you try to patch the CC service stub to the running product and check?
>
>
>
> On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Here we go, see attached files:
>
>
>
> ServiceUtils invokes the API
>
> CloudControllerClient, CloudControllerServiceImpl for implementation
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 9:45 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
I recompiled all jars, generated the zips and installed the system from the installer zip files, you think it is still necessary ?
(but I’ll try anyway - recompile service stub and patch)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Wednesday, June 04, 2014 10:58 AM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
Can you try to patch the CC service stub to the running product and check?
On Wed, Jun 4, 2014 at 11:18 PM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Here we go, see attached files:
>
>
>
> ServiceUtils invokes the API
>
> CloudControllerClient, CloudControllerServiceImpl for implementation
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Wednesday, June 04, 2014 9:45 AM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Here we go, see attached files:
ServiceUtils invokes the API
CloudControllerClient, CloudControllerServiceImpl for implementation
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Wednesday, June 04, 2014 9:45 AM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>> wrote:
I am sorry Martin, but it'll easy for me, if you can attach the code diff of CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
On Wed, Jun 4, 2014 at 10:14 PM, Nirmal Fernando <ni...@gmail.com>
wrote:
> I am sorry Martin, but it'll easy for me, if you can attach the code diff
> of CC component.
>
> Thanks.
>
>
> On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
>> Attached to the email,
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 10:20 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Can you please send a diff over?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> No, but I modeled / followed the deployCartridgeDefinition API which I
>> think passes a bean as parameter ?
>>
>>
>>
>> Is it required to define a bean ? Where and how ?
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 8:57 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Did you introduce a new bean class?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Actually,
>>
>>
>>
>> Below are all the changes I made to the wsdl for the new API, all I want
>> at this point is to pass a string parameter in the API:
>>
>>
>>
>> <wsdl:operation name="deployCompositeApplicationDefinition">
>>
>> <soap:operation
>> soapAction="urn:deployCompositeApplicationDefinition"
>> style="document"></soap:operation>
>>
>> <wsdl:input>
>>
>> <soap:body use="literal"></soap:body>
>>
>> </wsdl:input>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <soap:fault use="literal"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>>
>> </wsdl:fault>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidIaasProviderException">
>>
>> <soap:fault use="literal"
>> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>>
>> </wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>> <wsdl:operation name="deployCompositeApplicationDefinition">
>>
>> <soap12:operation
>> soapAction="urn:deployCompositeApplicationDefinition"
>> style="document"></soap12:operation>
>>
>> <wsdl:input>
>>
>> <soap12:body use="literal"></soap12:body>
>>
>> </wsdl:input>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <soap12:fault use="literal"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>>
>> </wsdl:fault>
>>
>> <wsdl:fault
>> name="CloudControllerServiceInvalidIaasProviderException">
>>
>> <soap12:fault use="literal"
>> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>>
>> </wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>> <wsdl:operation
>> name="deployCompositeApplicationDefinition">
>>
>> <http:operation
>> location="deployCompositeApplicationDefinition"></http:operation>
>>
>> <wsdl:input>
>>
>> <mime:content type="text/xml"
>> part="parameters"></mime:content>
>>
>> </wsdl:input>
>>
>> </wsdl:operation>
>>
>>
>>
>>
>>
>> <xs:element
>> name="deployCompositeApplicationDefinition">
>>
>> <xs:complexType>
>>
>> <xs:sequence>
>>
>> <xs:element minOccurs="0"
>> name="compositeApplicationConfig" nillable="true"
>> type="xs:string"></xs:element>
>>
>> </xs:sequence>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>>
>>
>>
>>
>>
>>
>> <wsdl:message
>> name="deployCompositeApplicationDefinitionRequest">
>>
>> <wsdl:part name="parameters"
>> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>>
>> </wsdl:message>
>>
>>
>>
>> <wsdl:operation
>> name="deployCompositeApplicationDefinition">
>>
>> <wsdl:input
>> message="ns:deployCompositeApplicationDefinitionRequest"
>> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>>
>> <wsdl:fault
>> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
>> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>>
>> <wsdl:fault
>> message="ns:CloudControllerServiceInvalidIaasProviderException"
>> name="CloudControllerServiceInvalidIaasProviderException"
>> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>>
>> </wsdl:operation>
>>
>>
>>
>>
>>
>> <xs:element
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <xs:complexType>
>>
>> <xs:sequence>
>>
>> <xs:element minOccurs="0"
>> name="InvalidCompositeApplicationDefinitionException" nillable="true"
>> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>>
>> </xs:sequence>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>>
>>
>> <wsdl:message
>> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>>
>> <wsdl:part name="parameters"
>> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>>
>> </wsdl:message>
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Tuesday, June 03, 2014 8:47 PM
>> *To:* dev@stratos.apache.org
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Yes,
>>
>>
>>
>> I did add the new api in the cloudcontroller.wsdl and regenerated the
>> stub (recompiled successfully all projects, including service-stub, etc …).
>> The error happens during run time when I invoke the API, are there any
>> debug traces I can turn on for axis which might help ?
>>
>>
>>
>>
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Tuesday, June 03, 2014 8:11 PM
>> *To:* dev@stratos.apache.org
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
>> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Marin,
>>
>> If you have changed the CloudController interface, you will need to use
>> the new cloud controller WSDL and re-generate the Cloud Controller service
>> stub code. This error can be a result of this. Let me know if you have not
>> done this so that I can give some guide lines.
>>
>>
>>
>> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> When I invoke my soap API I get the following exception, what ‘s the best
>> way to troublshoot / debug this ? What could be the issue ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
>> occurred while trying to invoke service method
>> deployCompositeApplicationDefinition
>> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>>
>> java.lang.ArrayIndexOutOfBoundsException: 1
>>
>> at
>> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>>
>> at
>> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>>
>> at
>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>>
>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>
>> at
>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>>
>> at
>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>>
>> at
>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>>
>> at
>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>
>> at
>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>
>> at
>> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>>
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>>
>> at
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>>
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>>
>> at
>> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>>
>> at
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>>
>> at
>> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>>
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>
>> at
>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>>
>> at
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>>
>> at
>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
>> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
>> occurred while trying to invoke service method
>> deployCompositeApplicationDefinition
>> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>>
>> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
>> service method deployCompositeApplicationDefinition
>>
>> at
>> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>>
>> at
>> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>>
>> at
>> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>>
>> at
>> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>>
>> at
>> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>>
>> at
>> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>>
>> at
>> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>>
>> at
>> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>
>>
>> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
>> *Sent:* Tuesday, June 03, 2014 2:30 PM
>> *To:* dev
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
>> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
>> Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Who knows about this group id? Who's suppose to set it to the topology?
>>
>>
>>
>> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru,
>>
>>
>>
>> As part of the grouping I need to add the groupId to a Member event (e.g.
>> MemberStartedEvent, MemberActivatedEvent).
>>
>> Currently I added the composite application data to the Topology by
>> generating a event and respective event processor (similar to Service)
>> which works fine for the autoscaler.
>>
>> However it seems that the CloudController uses its own registry and the
>> application info is missing when the topology is restored so it seems I
>> have to push the application data also to the cloud controller.
>>
>> The only interface available seems to be the soap api
>> (CloudControllerService.wsdl) which I would like to avoid (complicated and
>> time intensive). Is there a better way to push the data, can I use a
>> topology event, share the manager registry or do you have any other
>> recommendations ?
>>
>> In case the soap API is the only option, is there a better way to
>> generate the CloudControllerService .wsdl or do I have to manually edit in
>> my API changes ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Tuesday, May 20, 2014 9:15 AM
>> *To:* Isuru Haththotuwa
>> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Sounds good
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Tuesday, May 20, 2014 4:41 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>>
>> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Can you give me a quick json example to make sure I won’t misunderstand ?
>>
>> We would still be using the component JSON definitions as per your
>> proposal, since if we put everything inline it would be one large JSON
>> file. In the top level Composite Application Definition (which is the more
>> appropriate name for subscription Plan, as suggested by Lakmal), you can
>> refer the previously deployed groups by using the alias. So, the last
>> example that you have given in the doc would be a valid JSON sample. Hope
>> this is clear. Do let me know if its not.
>>
>>
>>
>>
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 11:33 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Lakmal,
>>
>>
>>
>> A few clarifications:
>>
>>
>>
>> I think based on the last proposal we can consolidate the definition of
>> components, dependencies in one definition and deploy all at once as
>> subscription plan, correct (simplifies the deployment) ?
>>
>>
>>
>>
>>
>> Yes, but as the first step, will have separate deployers for groups,
>> components and then composite application (which is you have mention
>> subscription plan).
>>
>>
>>
>> Also, Isuru pointed out that the term application might be confusing
>> so I suggested subscription plan (or something similar), do we still want
>> to go with it ?
>>
>>
>>
>>
>>
>> What i suggest to call it composite application.
>>
>>
>>
>> And composite application can refer, previous deployed group, components.
>> components can refer cartridge definitions. And dependancies (start order,
>> kill order) should be defined in the top level, which is in composite
>> application definition.
>>
>>
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Monday, May 19, 2014 10:58 AM
>> *To:* Martin Eppel (meppel); Lakmal Warusawithana
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Sorry, hit the send button to soon:
>>
>>
>>
>> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
>> checkin.
>>
>>
>>
>> Do we have already a planned date for 4.1 Milestone 1 ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Monday, May 19, 2014 10:52 AM
>> *To:* 'Lakmal Warusawithana'
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 9:45 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Lakmal,
>>
>>
>>
>> This sounds good to me, how do you suggest to coordinate the development.
>> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>>
>>
>>
>>
>>
>> No, lets work on master branch. Which part do you like to work on? Can
>> you start work on deploy these new definitions?
>>
>>
>>
>> Need to have persist model also. IsuruH, can you guys come up with it.
>>
>>
>>
>> Melan, can you start extent Composite Application create event?
>>
>>
>>
>> Lahiru can you start working on Composite Application monitor?
>>
>>
>>
>> Will target 4.1.0 Milestone 1.
>>
>>
>>
>> Is this look like a plan :)
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* Monday, May 19, 2014 2:06 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
>> Jayasingha; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Shall we start implementation, I can see following tasks.
>>
>> - Extending SM API for
>>
>>
>> - deploy components
>> - deploy group
>> - deploy composite application definition
>>
>>
>> - persist above deployment
>> - validation
>> - Extend SM for Composite Application creation in the topology
>> (earlier we had cluster create)
>> - Extend Auto Scaler (implementing Composite Application monitor)
>>
>>
>>
>> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
>> wrote:
>>
>> Thanks Martin, get clear idea now.
>>
>>
>>
>> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, Melan
>>
>>
>>
>> Based on our last night conversation I want to slightly modify the
>> previous grouping proposal:
>>
>>
>>
>> Instead of defining an application (misleading naming) we define a
>> subscription plan:
>>
>>
>>
>> The subscription defines the following:
>>
>> + grouping of components (or subscription_groups)
>>
>> + subscription_groups logically group subscribed cartridges and other
>> subscription groups (nested grouping) ( by means of referencing their alias)
>>
>> + dependencies between components (startup, termination, etc)
>>
>> + inline definition of subscription_groups in json doc (see example
>> further down)
>>
>>
>>
>> Going with this proposal I think we could leave current cartridge
>> subscription unchanged but tie the subscribed cartridges to a subscription
>> plan. Startup and termination dependencies will be checked by the
>> autoscaler.
>>
>>
>>
>> Please see also the section “*Subscription Plan”*in the shared google doc
>>
>>
>>
>>
>>
>> I would like to call it as "Composite Application" instead of
>> "Subscription Plan".
>>
>>
>>
>>
>>
>> What do you think ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>>
>>
>>
>>
>>
>> *Json example:*
>>
>>
>>
>> {
>>
>> "subscriptionPlanId": "subscriptionId",
>>
>> "alias": "subscription name",
>>
>> "components": [
>>
>> {
>>
>> "configGroup": {
>>
>> "alias": "group1",
>>
>> "subscribables": [
>>
>> "c1",
>>
>> "group2"
>>
>> ],
>>
>> "dependencies": {
>>
>> "startup_order": [
>>
>> {
>>
>> "key": "c1",
>>
>> "value": "c2"
>>
>> }
>>
>> ],
>>
>> "kill_behavior": "asymmetrical"
>>
>> }
>>
>> }
>>
>> },
>>
>> {
>>
>> "configGroup": {
>>
>> "alias": "group2",
>>
>> "subscribables": [
>>
>> "c3"
>>
>> ]
>>
>> }
>>
>> }
>>
>> ]
>>
>> }
>>
>>
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Saturday, May 17, 2014 2:44 PM
>> *To:* Isuru Haththotuwa
>>
>>
>> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
>> Warusawithana; Shaheedur Haque (shahhaqu)
>>
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Absolutely,
>>
>>
>>
>> We want to incorporate the feedback of all the experts but in the
>> meantime we can get started based on what we know and then gradually
>> enhance.
>>
>>
>>
>> IMHO it would also make sense to start developing the feature on a
>> separate branch (of RC3 tag ) and gradually improve the feature until we
>> feel it is ready to be incorporated into the mainline, this would allow us
>> to have something ready by the end of the month without destabilizing the
>> master branch ?
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Saturday, May 17, 2014 10:16 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
>> Warusawithana; Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> Thanks a lot for your efforts. Will try to have a look at the code before
>> the call.
>>
>> IMHO, before adding our changes, it is better to share the intended
>> design with the dev list and get feedback as well. We might need to come
>> with some sequence diagrams as well. I was planning to do that after we
>> have the call tomorrow. This is a crucial and a core change, plus a very
>> useful feature to Stratos, so better to get feedback from the experts in
>> the dev list. WDYT?
>>
>> I prepared some JSON definition referring your ones, in which I have only
>> one single JSON definition for the group subscription. My intention is to
>> re-use the Subscription model that we have in Stratos Manager and extend it
>> to support groups and dependencies. Lets discuss during the call, and
>> decide how to proceed.
>>
>>
>>
>> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, Melan
>>
>>
>>
>> Attached is the code I started implementing:
>>
>> 1. Added REST API for application deployment
>>
>> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
>> infrastructure)
>>
>>
>>
>> I also attached the sample JSON I used to test the REST API:
>>
>>
>>
>> Some of the code is experimental, e.g publishing the
>> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
>> messaging infrastructure and to be able to implement and test the backend
>> changes (application json) in the autoscaler (not there yet).
>>
>>
>>
>> I think there is still a bunch of things missing like an application
>> deployment / subscriptions manager (similar to deploying a cartridge, etc
>> …), but I am not entirely sure what you guys have in mind there.
>>
>> I do have some idea what should be done in the autoscaler to add
>> dependencies check, so I was thinking you could focus on the deployment
>> logic, messaging, etc … while I take a look at the autoscaler stuff.
>>
>> Anyway, this is just a suggestion, take a look and we can discuss more
>> details in our call.
>>
>>
>>
>> Regards
>>
>>
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Martin Eppel (meppel)
>> *Sent:* Friday, May 16, 2014 8:16 PM
>> *To:* Isuru Haththotuwa
>>
>>
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
>> Haque (shahhaqu); Melan Jayasingha
>>
>> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Isuru,
>>
>>
>>
>> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
>> time would be Sunday 8.30 AM IST)
>>
>>
>>
>> Regards
>>
>>
>>
>> Martin
>>
>>
>>
>> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
>> *Sent:* Friday, May 16, 2014 10:50 AM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
>> Haque (shahhaqu); Melan Jayasingha
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru,
>>
>>
>>
>> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
>> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>>
>> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
>> IST)
>>
>>
>>
>> How will we connect, I haven’t used Google hangout yet ?
>>
>> We can connect through either Google hangout/ Skype, whatever easier
>> for you.
>>
>>
>>
>> I noticed that emails are getting very slowly processed on dev@stratos
>> list so I am sending it also directly to you as well,
>>
>> I too noticed that your last mail got delayed by approx. one day.
>>
>>
>>
>> Btw, we are trying to get this feature ready (at least in some basic
>> form) by the end of the month
>>
>> I'm positive we can get an initial version working by the end of this
>> month, if all goes well.
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Thursday, May 15, 2014 9:10 PM
>> *To:* dev@stratos.incubator.apache.org
>> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>>
>>
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>> I'm in IST. Please suggest a convenient time for you. I would like to
>> clarify several things on the proposal and the concepts you brought in,
>> simplify it further if possible, and decide how to proceed.
>>
>>
>>
>> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, see inline
>>
>>
>>
>> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
>> Haththotuwa
>> *Sent:* Wednesday, May 14, 2014 1:12 PM
>> *To:* Martin Eppel (meppel)
>> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
>> Haque (shahhaqu)
>>
>>
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin,
>>
>>
>>
>> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Isuru, Shaheed
>>
>>
>>
>>
>>
>> Ad “kill behaviour”:
>> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”,
>> “kill-none” (btw, was left out accidentally, thanks for pointing it out),
>>
>>
>>
>> Ad single definition:
>>
>>
>>
>> Makes sense too, however we should still consider that the json
>> definition can become fairly complex when multiple components and sub
>> components are defined, that’s the reason I think it would still make sense
>> to keep it modular, at least as an option.
>>
>> +1
>>
>> Martin: +1
>>
>> (btw, I assume you suggest instead of defining each component in
>> separate files we aggregate them in the same application json document,
>> but not like a “in-line” definition of each component?)
>>
>> Exactly . This is fine as the first step IMHO.
>>
>>
>>
>> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>>
>> Does this include other definitions like cartridge info definition,
>> deployment policy, autoscaling definition, etc ... ?
>>
>> No. I agree that if we do this, it will be an unmanageably large JSON
>> definition. We can refer to the relevant policies (which are already
>> deployed) in the service group definition. The service group should contain
>> the group specific details IMHO, as the dependencies, startup orders, kill
>> behaviours, etc. WDYT?
>>
>>
>>
>> Martin: +1 (see my example json definitions)
>>
>>
>>
>> We could provide eventually both options, in-line with all components
>> defined in one document (simple configurations) or modular, where
>> components are deployed (un-deployed) individually (for large, complex
>> configurations).
>>
>>
>>
>> In the shared document I provided a full example of the json definition
>> of an application, and just adding a few components seemed to make the
>> document fairly large or complex (see chapter “Json example”).
>>
>> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
>> we can decide on the next steps.
>>
>>
>>
>> Martin: sounds good, Friday would be Thursday my time, which time zone
>> are you in ?
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Martin
>>
>>
>>
>>
>> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>>
>>
>>
>>
>>
>> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
>> *Sent:* Tuesday, May 13, 2014 6:08 AM
>> *To:* dev@stratos.incubator.apache.org
>> *Cc:* Lakmal Warusawithana
>>
>>
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> Hi Martin, Shaheedur and all,
>>
>> Went through the shared document for the proposal. good work! One small
>> suggestion, additionally to deploying component JSON definitions one by
>> one, shall we allow to deploy a one single definition (that will have all
>> the dependencies and other information defined) to be deployed? WDYT? IMHO
>> this will be more user friendly.
>>
>>
>>
>>
>>
>> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
>> shahhaqu@cisco.com> wrote:
>>
>> A couple of points on “kill behaviour”:
>>
>>
>>
>> · I don’t think the terms “asymmetric” and “symmetric” capture
>> the behaviours I had in mind. For me there were two use cases:
>>
>> o Assuming A depends on B and B depends on C
>>
>> o And B fails
>>
>> o Use case #1 is that only A should be killed and allowed to restart.
>> We could call this kill behaviour “kill-dependents” since A is dependent on
>> B. This fits the standard n-tier architecture where A is the front end, B
>> is the application logic and C is the database.
>>
>> o Use case #2 is that both A and C should be killed and allowed to
>> restart. We could call this “kill-all”. This fits the use case of a poorly
>> behaved application where everything needs a restart on any failure.
>>
>> · Did we accidentally drop the third value, i.e. “kill-none”, or
>> did we somehow convince ourselves this was not needed? I’m actually tending
>> towards the latter since if kill-none were a usable value, it seems to
>> contradict that there was a dependency in the first place.
>>
>> +1.
>>
>>
>>
>> Thanks, Shaheed
>>
>>
>>
>> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> *Sent:* 07 May 2014 06:48
>>
>>
>> *To:* dev@stratos.incubator.apache.org
>> *Cc:* Shaheedur Haque (shahhaqu)
>> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>>
>>
>>
>>
>> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>>
>> Hi Lakmal,
>>
>> See inline "Martin:"
>>
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>>
>> Sent: Tuesday, May 06, 2014 3:25 AM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Hi Martin,
>>
>> Thanks for adding content, I had offline discussion with Imesh, Lahiru
>> and IsuruH on this, we can start implementing step by step. will do this by
>> milestone by milestone approach.
>>
>> Further I think we can used same topology topic to create this model.
>> With this root node of the topology will be application. From the auto
>> scaler we can implement application monitor to handle this. service
>> clusters will be sub set of application.
>>
>> Martin: +1, dependencies have to be included
>>
>>
>> As the first step we reused (partitions, auto scaling policy, deployment
>> policy, cartridge definitions) existing definitions to define application
>> with new requirement like grouping, dependency, start order ..etc. Shall we
>> create sample json ( with sample auto scaling policies, deployment
>> policies, cartridges ..etc) to get some idea for the complexity? (still I
>> am figuring out some definition like subscribable and some real sample json)
>>
>> Martin: +1, Actually, the json example I posted on Google docs assumes
>> that we'll be reusing all the existing artifacts (like cartridge definition
>> etc ...). Cartridges, policies, componenents are referenced by name so each
>> of these have to be deployed separately (as they are today) and when the
>> application is deployed it will reference the deployed artifacts - what do
>> you think ?
>> We'll might have to add some extra properties - I'll work on a complete
>> example and post the progress on Google Doc.
>>
>>
>>
>> Then we should divide each work item and start implementing. Ideally in
>> the milestone one should not break existing, but reused them. Then later we
>> can identifying what need to improve/change existing components.
>>
>> Martin: +1, would it speed up / simply implementation if we start working
>> on a branch ? Also, I totally agree, all code change should be
>> complementary without breaking any existing features
>>
>>
>> Later this week we can have google hangout session to discuss further on
>> this.
>>
>> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we
>> can do a hangout either today, tomorrow or then again next week Wednesday.
>>
>>
>>
>> Seems like we have to wait till next Friday, next week, Wednesday and
>> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
>> identify initial task and will post here.
>>
>>
>> Lets have a hangout at a convenient time to discuss the next steps.
>>
>>
>> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Works for me, I’ll add the content
>>
>> Thanks
>>
>> Martin
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Sunday, May 04, 2014 8:56 PM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Hi Martin,
>>
>> Thanks for detail description. We can create wiki space , but IMHO its
>> easy if we can use public google doc. I have created [1] shall we add our
>> proposal into it. May be we need some hangout session to discuss further on
>> this.
>>
>> [1]
>> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>>
>>
>> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi Lakmal,
>>
>> I summarized and formatted the discussion points from the previous
>> email(s) thread for better readability and reference.
>> Btw, what is the appropriate way going forward to elevate the discussion
>> to a feature request (JIRA) ? Also, how do we typically discuss feature
>> requests like this, the email format makes it somewhat difficult to read
>> and it is easy to overlook posted discussion points ? Is there a wiki to
>> post and discuss proposals like this ?
>>
>> Thanks
>>
>> Martin
>>
>> Grouping of Cartridges / Services:
>>
>> 1. Application model
>> 1.1 General model
>> See proposal below
>> 1.2 Json
>> see proposal below
>>
>> 2. Application deployment
>> 2.1 Deployment
>> + application deployment (deploy)
>> + application removal (undeploy)
>> 2.2 Subscription
>> + subscription (subscribe)
>> + cancelation of subscription (unsubscribe)
>>
>> 2. Distributing Application deployment
>> 2.1 Publishing application deployment
>> + new topic to publish application events, map, status
>> 2.2 Subscribing to published application deployment
>> + Autoscaler
>>
>> 3. Dependency calculation
>> 3.1 Building static dependency tree
>> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
>> + Dependency tree determines all dependencies,
>> + symmetrical / asymmetrical dependencies
>> + Dependency checks member status for all dependencies
>> successful for at least one member in ACTIVE state, failed
>> for no member in ACTIVE state
>> 3.1.1.1 Enhancing autoscaler rules
>> + Integrate with
>> mincheck.drl to control instance creation,
>> termination rule to control instance termination
>>
>>
>> 4. Application event model
>> + provides state information for application, application components
>> like
>> Up, Down, partial up
>>
>>
>>
>> From: Martin Eppel (meppel)
>> Sent: Thursday, May 01, 2014 3:26 PM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: RE: [Discuss] Grouping of services (cartridges)
>>
>> Hi Lakmal at al
>>
>> Here is a first draft for an application model as a starting point, it
>> builds on the previous discussions (and probably leaves out a few things):
>> Also, to add to Shaheeds point there is probably an event model around
>> this as well.
>>
>> Definitions:
>> + composite structure
>> + A component is sectioned into description, components, configuration,
>> dependencies.
>> + Components are typed (application, subscribables, scalable)
>> + Application is also component
>>
>> + description ... properties (like name, type)
>> + components ... children, nested
>> + configuration ... properties
>> + dependencies ... puts immediate components (children) in relation
>>
>> Generic Component format:
>> + description (component specific properties)
>> + components (by reference)
>> + configuration
>> - described through properties
>> + properties can be grouped, group can be referenced
>> + dependencies
>> - described through properties
>>
>>
>> In json:
>>
>> Generic Component:
>>
>> Component:
>>
>> {
>> "description": [
>> ],
>> "configuration":[
>> ],
>> "components":[
>> ],
>> "dependencies":[
>> ]
>> }
>>
>>
>> Application:
>>
>> {
>> "description": [
>> {"name":"application name"},
>> {"types":["application", "subscribable"]}
>> ],
>> "configuration":[
>> {"deployment":"deployment_policy_name"}
>> ],
>> "components":[
>> {"subscribables": ["subscribable name",
>> "subscribable name",
>> "subscribable name"]}
>> ],
>> "dependencies":[
>> {"startup_order": [
>> {"dependant name": "subscribable name"},
>> {"dependant name": "subscribable name"}
>> ]},
>> {"kill_behavior": "symmetrical or asymmetrical"}
>> ]
>> }
>>
>> Subscribable (aka groups):
>>
>> {
>> "description": [
>> {"name":"component name"},
>> {"types":["subscribable"]}
>> ],
>> "configuration":[
>> ],
>> "components":[
>> {"subscribables": ["subscribable name",
>> "subscribable name",
>> "subscribable name"]}
>> ],
>> "dependencies":[
>> {"startup_order": [
>> {"dependant name": "subscribable name"},
>> {"dependant name": "subscribable name"}
>> ]},
>> {"kill_behavior": "symmetrical / asymmetrical"}
>> ]
>> }
>>
>> Cartridge Component:
>> {
>> "description": [
>> {"name":"component name"},
>> {"types":["subscribable", "scalable"]}
>> ],
>> "configuration":[
>> {"scaling":"autoscaling policy name"},
>> {"info":"cartridge info"}
>> ],
>> "components":[
>> ],
>> "dependencies":[
>> ]
>> }
>>
>>
>> Thanks
>>
>> Martin
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Thursday, May 01, 2014 1:18 AM
>> To: Shaheedur Haque (shahhaqu)
>> Cc: dev@stratos.incubator.apache.org
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
>> Hi,
>>
>> Sorry for top-posting, but one additional thing that I feel should be
>> considered is a minimal set of lifecycle events for the group; for example:
>>
>> - "all members of a group are now active", aka "group active".
>>
>> - "at least one member of an existing group has failed", aka "group down".
>>
>> This is clearly an add-on to the core functionality but is needed to
>> round the feature out.
>>
>>
>> +1
>>
>> Thanks, Shaheed
>>
>>
>> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
>> Hi Martin,
>>
>> With the current implementation I think and agree its better to go with
>> json as defining configuration data. Summarizing proposal and the
>> discussion I can think following data model. Here I am thinking
>> implementing single configuration data (application deployment definition)
>> model, which can define full configuration of an composite application. we
>> should take it down to several milestones. we can start on enhancing
>> backend services in initial milestone and later will come with aggregating
>> them. All of your thoughts welcome.
>>
>> Can you come up with some definition format. (may be json format). Also
>> see my inline comment.
>>
>> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi Lakmal,
>>
>> Below are ideas on how to break up the feature into a high level ToDo
>> list (with some suggestions and questions) :
>>
>> + Defining configuration data format – for example json or chef / recipe
>> ++ would using recipes affect all configuration data or is it
>> restricted to the grouping / dependency configuration only ?
>>
>> IMHO, its may easy with json, considering current existing services.
>>
>>
>> + Adding the new model to stratos manager and integrate with existing
>> model:
>> ++ Group model (nested groups, group properties)
>> ++ Dependency model
>> ++ in the current model we have a Subscriptions and Deployment
>> manager, I think we need something similar to handle the groups which in
>> turn invokes deployment / subscriptions manager
>>
>>
>> +1
>>
>> + Defining runtime dependency conditions:
>> ++ Calculating the dependency for a cartridge (service) dynamically
>> by walking the dependency tree and checking the state of dependencies (e.g.
>> member state)
>>
>>
>> +1
>>
>> + Integrate the dynamic aspect of the dependency model (checking the
>> conditions of dependencies and act accordingly):
>> ++ As of now I think it has to be integrated with the autoscaler as
>> it the entity which controls instances (members) and checks their states
>>
>> Yes, we need to enhance auto scaler
>>
>> ++ How is dependency retrieved from configuration data (e.g.
>> published through topology, beans, etc ….) ?
>>
>> We need to think on more this. IMO, we may need another topic
>> (application deployment definition) to retrieved the config data. SM can
>> define this based on provide definition, auto scaler and who ever need can
>> get that. Topology is maintaining current running state (currently its for
>> a service, we need to enhance it to application), so we can used
>> "application deployment definition" data to check with current state.
>>
>> ++ Integrate dependency checks into the runtime portion of the
>> autoscaler, e.g. add the checks to the monitor functionality of a
>> ClusterMonitor with respective actions (spawning / terminating instances)
>>
>>
>> I think we can coupled auto scaling policy with individual
>> cartridges/services.
>>
>> What do you think ?
>>
>>
>> We may need to think about deployment policy, I think with the
>> application deployment definition (with some coupling ,like some cartridges
>> may need to spin up same partition..etc). May be we are redefining
>> deployment policy with application deployment definition. (we may obsolete
>> deployment policy)
>>
>> Thanks
>>
>> Martin
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Wednesday, April 30, 2014 4:30 AM
>>
>> To: dev@stratos.incubator.apache.org
>> Cc: Shaheedur Haque (shahhaqu)
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Hi Martin,
>>
>> These use cases are very valid, and we should integrate them into 4.1.0.
>> Will go through in detail and see how we can incorporate into Stratos.
>> I have one question, why do we need hierarchical grouping, (I mean group
>> inside a group) and use case? Can't we have flat?
>>
>> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi Lakmal
>>
>> Below is a further enhanced proposal to add grouping to apache stratos:
>>
>> In a nutshell:
>>
>> By grouping cartridges together we can define characteristics which apply
>> to all cartridges alike most importantly it will allow us also to define
>> dependencies between cartridges. Groups should also be flexible enough to
>> not only contain single cartridges but also groups, allowing a hierarchical
>> structure.
>>
>> Dependencies are defined at the group level, describing the dependency
>> relationship of immediate children.
>> Other group specific properties could be defined which will apply to all
>> immediate group members.
>>
>> The subscription model will be extended to subscribe to a group in
>> addition to a cartridge (either or). To generalize we added the concept of
>> a Subscribable which can be either a group or cartridge.
>>
>> Since a Subscribale (or more specifically a group) doesn’t have scaling
>> characteristics, it seemed appropriate to add the concept of Scalable,
>> describing the scalable nature of a cartridge.
>>
>> Below is a more detailed description and a corresponding diagram:
>>
>> “Class diagram” of the logical model showing the existing description
>> files in the Diadem model, and the proposed changes.
>>
>> Description: see attachment Slide1.png
>>
>> Some notes on the new dependency system:
>>
>> * A Group can be subscribed or a Scalable can be subscribed, whereas
>> today, a Cartridge is subscribed.
>> o The new Scalable entity is needed because currently
>> Subscriptions own autoscale policies, but that doesn’t work when
>> a Group contains more than one Cartridge, because we want one
>> autoscale policy per Cartridge. However we can’t couple the
>> autoscale policy to the Cartridge because it may be re-used in
>> other Subscriptions where a different autoscale policy is desired
>> * children is the set of Subscribables in the Group.
>> o This supports hierarchical Groups.
>> * collocate says “these Children must be physically next to each other”
>> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
>> but B doesn’t point to A.
>> o These pairs define a DAG, which we use to perform a topological
>> sort of the children and get a /partial/ ordering.
>> o Any member of “children” not in a pair is a singleton started in
>> parallel to everything else.
>> * The killBehaviour says that when node X dies, we must kill:
>> o Everything else (for applications where the loss of any member
>> of “children” cannot be recovered with less impact
>> o Nothing else (for applications where any element can be restarted)
>> o Or just the things “upstream” in the startup order (for
>> conventional tiered applications)
>>
>> We think this describes all the use-cases we’ll ever see with the
>> exception of things like “n instances of A depends on B”; that can be
>> considered in the future as an enhancement.
>>
>> Startup use cases considered
>>
>> * All start in parallel: don’t specify any pairs => everything is
>> equivalent => all start in parallel
>> * All start one-after-another (total order): specify a set of pairs
>> such that there is a single contiguous list covering all the nodes,
>> e.g. A -> B -> C -> D -> E
>> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
>> -> Database], [Logging]]
>>
>> Kill use cases considered
>>
>> * One dies => all die: set killBehaviour to KillAll
>> * One dies => nothing happens: set killBehaviour to killNone
>> * One dies => its dependancies die (aka “restart from here”): specify
>> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
>> -> Database], [Logging]], Database dies => MiddleTier is restarted,
>> Logging untouched.
>>
>> Example instantiation
>>
>> Description: see attachment Slide2.png
>>
>> Thanks
>>
>> Martin
>>
>>
>> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>> Sent: Sunday, March 30, 2014 9:09 PM
>>
>> To: dev@stratos.incubator.apache.org
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>>
>>
>> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> I totally agree, I am also not in favor of complicating existing
>> components (e.g. autoscaler).
>>
>> However, I am not sure what the alternative might be to solve the
>> requirements (e.g. item 1) mentioned below.
>> The suggestion I made to enhance the autoscaler / rules are based on my
>> understanding that the autoscaler already handles similar actions (e.g. VM
>> startup, scaling and termination).
>> It seems to me a logical extension to add additional knowledge to handle
>> the boot sequencing, dependent scaling and dependent termination to the
>> autoscaler defined by optional properties in the autoscaler policy.
>>
>> From my current point of view, CAMP seems a fairly complex specification
>> which might require quite some changes to adopt.
>>
>> Sorry I could not go through CAMP in detail. Should spend sometime.
>>
>>
>> I do agree that alternatives might exist and should be discussed !?
>> Alternatives ?
>>
>> +1 for alternatives, we should not take CAMP as it is, we should see how
>> we can match with existing Stratos workflow IMO.
>>
>>
>> Thanks
>>
>> Martin
>>
>> From: damitha kumarage [mailto:damitha23@gmail.com]
>> Sent: Sunday, March 30, 2014 7:59 AM
>> To: dev@stratos.incubator.apache.org
>>
>> Subject: Re: [Discuss] Grouping of services (cartridges)
>>
>> Please see my inline comment,
>>
>> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
>> wrote:
>> Hi Martin,
>> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
>> wrote:
>> Hi,
>>
>> I think this property will be a quite useful piece to solve the grouping
>> issue:
>>
>> I also would like to suggest to add the serviceGroup to the topology map
>> (in case it is not yet available in the topology map). This will help to
>> tie together cartridges (or services) in the autoscaler and , for example
>> enable synchronized auto scaling behavior of services within a service
>> group, like synchronized scaling, sequenced boot up, etc ….
>>
>> In addition the autoscaler should be enhanced to add additional (but
>> optional properties) in the auto scaling policy related to a service group
>> to govern the respective auto scaling behavior.
>>
>> For example, related properties should identify a service group and other
>> related properties to define dependencies between the various cartridges in
>> the service group like boot sequence, scale up / down ratios, termination
>> dependencies, etc … . The property set (or json structure ) should be
>> fairly flexible as we are just about to explore this new feature and should
>> be easily expandable.
>>
>> I would think that these additions will also prove useful to integrate in
>> the long term with CAMP (or other spec) but will help to solve more
>> immediate requirements
>>
>>
>> Yes, these are very valid points in coming up with a proper grouping
>> architecture for services. Thank you for bringing them up.
>>
>> As I understand, what Sajith has done here is enabling static grouping of
>> services, by using a property for that in the cartridge deployment time.
>> What we are trying to achieve in long term is dynamic grouping of services,
>> so that we can group any available service at runtime seamlessly, according
>> to CAMP specification (or some other suitable way).
>>
>> I see three things here
>> 1) Grouping and resolve inter-dependancies between cartridges.
>> 2) Composite Artifact deployment (May be if required, according to the
>> above grouping and inter-dependancy( of cartridges. There can also be
>> intra-dependancies inside a cartridge in case of artifact deployment)
>> 3) Improved nonitoring and health check of above intricacies.
>> Instead of adopt CAMP to solve above, I think it is better to discuss and
>> find ways that fits most naturally to the existing Stratos
>> architecture(Unless CAMP is widely adopted and we are compelled to
>> adhere). Is there a way we can solve this without doing major changes to
>> existing components? For example without complicating autoscaler
>> policies/logic as suggested by Martin above?
>> Damitha
>>
>>
>>
>>
>> --
>> __________________________________________________________________
>> Damitha Kumarage
>> http://people.apache.org/
>> __________________________________________________________________
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Thanks and Regards,
>>
>> Isuru H.
>>
>> +94 716 358 048
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Thanks and Regards,
>>
>> Isuru H.
>>
>> +94 716 358 048
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>> --
>>
>> Lakmal Warusawithana
>>
>> Director - Cloud Architecture; WSO2 Inc.
>>
>> Mobile : +94714289692
>>
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>>
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
I am sorry Martin, but it'll easy for me, if you can attach the code diff
to CC component.
Thanks.
On Wed, Jun 4, 2014 at 10:07 PM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Attached to the email,
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 10:20 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Can you please send a diff over?
>
>
>
> On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Attached to the email,
Thanks
Martin
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Tuesday, June 03, 2014 10:20 PM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
Can you please send a diff over?
On Wed, Jun 4, 2014 at 10:46 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> No, but I modeled / followed the deployCartridgeDefinition API which I
> think passes a bean as parameter ?
>
>
>
> Is it required to define a bean ? Where and how ?
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 8:57 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Did you introduce a new bean class?
>
>
>
> On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
No, but I modeled / followed the deployCartridgeDefinition API which I think passes a bean as parameter ?
Is it required to define a bean ? Where and how ?
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Tuesday, June 03, 2014 8:57 PM
To: dev
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org<ma...@stratos.apache.org>
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
Hi Martin,
Did you introduce a new bean class?
On Wed, Jun 4, 2014 at 9:19 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Actually,
>
>
>
> Below are all the changes I made to the wsdl for the new API, all I want
> at this point is to pass a string parameter in the API:
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap:operation>
>
> <wsdl:input>
>
> <soap:body use="literal"></soap:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation name="deployCompositeApplicationDefinition">
>
> <soap12:operation
> soapAction="urn:deployCompositeApplicationDefinition"
> style="document"></soap12:operation>
>
> <wsdl:input>
>
> <soap12:body use="literal"></soap12:body>
>
> </wsdl:input>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
>
> </wsdl:fault>
>
> <wsdl:fault
> name="CloudControllerServiceInvalidIaasProviderException">
>
> <soap12:fault use="literal"
> name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
>
> </wsdl:fault>
>
> </wsdl:operation>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <http:operation
> location="deployCompositeApplicationDefinition"></http:operation>
>
> <wsdl:input>
>
> <mime:content type="text/xml"
> part="parameters"></mime:content>
>
> </wsdl:input>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="deployCompositeApplicationDefinition">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="compositeApplicationConfig" nillable="true"
> type="xs:string"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
>
>
>
>
> <wsdl:message
> name="deployCompositeApplicationDefinitionRequest">
>
> <wsdl:part name="parameters"
> element="ns:deployCompositeApplicationDefinition"></wsdl:part>
>
> </wsdl:message>
>
>
>
> <wsdl:operation
> name="deployCompositeApplicationDefinition">
>
> <wsdl:input
> message="ns:deployCompositeApplicationDefinitionRequest"
> wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
>
> <wsdl:fault
> message="ns:CloudControllerServiceInvalidIaasProviderException"
> name="CloudControllerServiceInvalidIaasProviderException"
> wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
>
> </wsdl:operation>
>
>
>
>
>
> <xs:element
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <xs:complexType>
>
> <xs:sequence>
>
> <xs:element minOccurs="0"
> name="InvalidCompositeApplicationDefinitionException" nillable="true"
> type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
>
> </xs:sequence>
>
> </xs:complexType>
>
> </xs:element>
>
>
>
> <wsdl:message
> name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
>
> <wsdl:part name="parameters"
> element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
>
> </wsdl:message>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, June 03, 2014 8:47 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Yes,
>
>
>
> I did add the new api in the cloudcontroller.wsdl and regenerated the stub
> (recompiled successfully all projects, including service-stub, etc …). The
> error happens during run time when I invoke the API, are there any debug
> traces I can turn on for axis which might help ?
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, June 03, 2014 8:11 PM
> *To:* dev@stratos.apache.org
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Marin,
>
> If you have changed the CloudController interface, you will need to use
> the new cloud controller WSDL and re-generate the Cloud Controller service
> stub code. This error can be a result of this. Let me know if you have not
> done this so that I can give some guide lines.
>
>
>
> On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> When I invoke my soap API I get the following exception, what ‘s the best
> way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Actually,
Below are all the changes I made to the wsdl for the new API, all I want at this point is to pass a string parameter in the API:
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap:operation>
<wsdl:input>
<soap:body use="literal"></soap:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<soap12:operation soapAction="urn:deployCompositeApplicationDefinition" style="document"></soap12:operation>
<wsdl:input>
<soap12:body use="literal"></soap12:body>
</wsdl:input>
<wsdl:fault name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException"></soap12:fault>
</wsdl:fault>
<wsdl:fault name="CloudControllerServiceInvalidIaasProviderException">
<soap12:fault use="literal" name="CloudControllerServiceInvalidIaasProviderException"></soap12:fault>
</wsdl:fault>
</wsdl:operation>
<wsdl:operation name="deployCompositeApplicationDefinition">
<http:operation location="deployCompositeApplicationDefinition"></http:operation>
<wsdl:input>
<mime:content type="text/xml" part="parameters"></mime:content>
</wsdl:input>
</wsdl:operation>
<xs:element name="deployCompositeApplicationDefinition">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="compositeApplicationConfig" nillable="true" type="xs:string"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="deployCompositeApplicationDefinitionRequest">
<wsdl:part name="parameters" element="ns:deployCompositeApplicationDefinition"></wsdl:part>
</wsdl:message>
<wsdl:operation name="deployCompositeApplicationDefinition">
<wsdl:input message="ns:deployCompositeApplicationDefinitionRequest" wsaw:Action="urn:deployCompositeApplicationDefinition"></wsdl:input>
<wsdl:fault message="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException" name="CloudControllerServiceInvalidCompositeApplicationDefinitionException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:fault>
<wsdl:fault message="ns:CloudControllerServiceInvalidIaasProviderException" name="CloudControllerServiceInvalidIaasProviderException" wsaw:Action="urn:deployCompositeApplicationDefinitionCloudControllerServiceInvalidIaasProviderException"></wsdl:fault>
</wsdl:operation>
<xs:element name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="InvalidCompositeApplicationDefinitionException" nillable="true" type="ax219:InvalidCompositeApplicationDefinitionException"></xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<wsdl:message name="CloudControllerServiceInvalidCompositeApplicationDefinitionException">
<wsdl:part name="parameters" element="ns:CloudControllerServiceInvalidCompositeApplicationDefinitionException"></wsdl:part>
</wsdl:message>
From: Martin Eppel (meppel)
Sent: Tuesday, June 03, 2014 8:47 PM
To: dev@stratos.apache.org
Cc: dev@stratos.incubator.apache.org; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com [mailto:isuruh@wso2.com] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org
Cc: dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Yes,
I did add the new api in the cloudcontroller.wsdl and regenerated the stub (recompiled successfully all projects, including service-stub, etc …). The error happens during run time when I invoke the API, are there any debug traces I can turn on for axis which might help ?
From: isuruh@wso2.com [mailto:isuruh@wso2.com] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, June 03, 2014 8:11 PM
To: dev@stratos.apache.org
Cc: dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Marin,
If you have changed the CloudController interface, you will need to use the new cloud controller WSDL and re-generate the Cloud Controller service stub code. This error can be a result of this. Let me know if you have not done this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Isuru Haththotuwa <is...@apache.org>.
Hi Marin,
If you have changed the CloudController interface, you will need to use the
new cloud controller WSDL and re-generate the Cloud Controller service stub
code. This error can be a result of this. Let me know if you have not done
this so that I can give some guide lines.
On Wed, Jun 4, 2014 at 8:21 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> When I invoke my soap API I get the following exception, what ‘s the
> best way to troublshoot / debug this ? What could be the issue ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
>
> java.lang.ArrayIndexOutOfBoundsException: 1
>
> at
> org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
>
> at
> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
>
> at
> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
>
> at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
>
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>
> at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
>
> at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
>
> at
> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
>
> at
> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>
> at
> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
>
> at
> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
>
> at
> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
>
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
>
> at
> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
>
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
>
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
>
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR
> {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception
> occurred while trying to invoke service method
> deployCompositeApplicationDefinition
> {org.apache.stratos.rest.endpoint.services.ServiceUtils}
>
> org.apache.axis2.AxisFault: Exception occurred while trying to invoke
> service method deployCompositeApplicationDefinition
>
> at
> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
>
> at
> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
>
> at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
>
> at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
>
> at
> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
>
> at
> org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
>
> at
> org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
>
> at
> org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>
>
> *From:* Nirmal Fernando [mailto:nirmal070125@gmail.com]
> *Sent:* Tuesday, June 03, 2014 2:30 PM
> *To:* dev
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal
> Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: adding the groupId to a member event ... RE: [Discuss]
> Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Who knows about this group id? Who's suppose to set it to the topology?
>
>
>
> On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
>
> --
>
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
>
>
> Blog: http://nirmalfdo.blogspot.com/
>
RE: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
When I invoke my soap API I get the following exception, what ‘s the best way to troublshoot / debug this ? What could be the issue ?
Thanks
Martin
TID: [0] [STRATOS] [2014-06-04 02:30:40,030] ERROR {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver}
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.axis2.databinding.utils.BeanUtil.deserialize(BeanUtil.java:639)
at org.apache.axis2.rpc.receivers.RPCUtil.processRequest(RPCUtil.java:153)
at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:206)
at org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [STRATOS] [2014-06-04 02:30:40,039] ERROR {org.apache.stratos.rest.endpoint.services.ServiceUtils} - Exception occurred while trying to invoke service method deployCompositeApplicationDefinition {org.apache.stratos.rest.endpoint.services.ServiceUtils}
org.apache.axis2.AxisFault: Exception occurred while trying to invoke service method deployCompositeApplicationDefinition
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
at org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.deployCompositeApplicationDefinition(CloudControllerServiceStub.java:389)
at org.apache.stratos.manager.client.CloudControllerServiceClient.deployCompositeApplicationDefinition(CloudControllerServiceClient.java:118)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployCompositeApplicationDefinition(ServiceUtils.java:227)
at org.apache.stratos.rest.endpoint.services.ServiceUtils.deployApplication(ServiceUtils.java:198)
at org.apache.stratos.rest.endpoint.services.StratosAdmin.deployApplicationDefinition(StratosAdmin.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
From: Nirmal Fernando [mailto:nirmal070125@gmail.com]
Sent: Tuesday, June 03, 2014 2:30 PM
To: dev
Cc: dev@stratos.incubator.apache.org; Isuru Haththotuwa; Lakmal Warusawithana; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: adding the groupId to a member event ... RE: [Discuss] Grouping of services (cartridges)
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
As part of the grouping I need to add the groupId to a Member event (e.g. MemberStartedEvent, MemberActivatedEvent).
Currently I added the composite application data to the Topology by generating a event and respective event processor (similar to Service) which works fine for the autoscaler.
However it seems that the CloudController uses its own registry and the application info is missing when the topology is restored so it seems I have to push the application data also to the cloud controller.
The only interface available seems to be the soap api (CloudControllerService.wsdl) which I would like to avoid (complicated and time intensive). Is there a better way to push the data, can I use a topology event, share the manager registry or do you have any other recommendations ?
In case the soap API is the only option, is there a better way to generate the CloudControllerService .wsdl or do I have to manually edit in my API changes ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Tuesday, May 20, 2014 9:15 AM
To: Isuru Haththotuwa
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sounds good
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, May 20, 2014 4:41 AM
To: Martin Eppel (meppel)
Cc: Lakmal Warusawithana; dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Can you give me a quick json example to make sure I won’t misunderstand ?
We would still be using the component JSON definitions as per your proposal, since if we put everything inline it would be one large JSON file. In the top level Composite Application Definition (which is the more appropriate name for subscription Plan, as suggested by Lakmal), you can refer the previously deployed groups by using the alias. So, the last example that you have given in the doc would be a valid JSON sample. Hope this is clear. Do let me know if its not.
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 11:33 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Lakmal,
A few clarifications:
I think based on the last proposal we can consolidate the definition of components, dependencies in one definition and deploy all at once as subscription plan, correct (simplifies the deployment) ?
Yes, but as the first step, will have separate deployers for groups, components and then composite application (which is you have mention subscription plan).
Also, Isuru pointed out that the term application might be confusing so I suggested subscription plan (or something similar), do we still want to go with it ?
What i suggest to call it composite application.
And composite application can refer, previous deployed group, components. components can refer cartridge definitions. And dependancies (start order, kill order) should be defined in the top level, which is in composite application definition.
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:58 AM
To: Martin Eppel (meppel); Lakmal Warusawithana
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Sorry, hit the send button to soon:
Ok, I’ll work on the deployment. I’ll send you guys the code as patch to checkin.
Do we have already a planned date for 4.1 Milestone 1 ?
Thanks
Martin
From: Martin Eppel (meppel)
Sent: Monday, May 19, 2014 10:52 AM
To: 'Lakmal Warusawithana'
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 9:45 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
This sounds good to me, how do you suggest to coordinate the development. Will it make sense to create a branch (based on RC3 ?) to develop on ?
No, lets work on master branch. Which part do you like to work on? Can you start work on deploy these new definitions?
Need to have persist model also. IsuruH, can you guys come up with it.
Melan, can you start extent Composite Application create event?
Lahiru can you start working on Composite Application monitor?
Will target 4.1.0 Milestone 1.
Is this look like a plan :)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Monday, May 19, 2014 2:06 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Isuru Haththotuwa; Melan Jayasingha; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Shall we start implementation, I can see following tasks.
* Extending SM API for
* deploy components
* deploy group
* deploy composite application definition
* persist above deployment
* validation
* Extend SM for Composite Application creation in the topology (earlier we had cluster create)
* Extend Auto Scaler (implementing Composite Application monitor)
On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>> wrote:
Thanks Martin, get clear idea now.
On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Based on our last night conversation I want to slightly modify the previous grouping proposal:
Instead of defining an application (misleading naming) we define a subscription plan:
The subscription defines the following:
+ grouping of components (or subscription_groups)
+ subscription_groups logically group subscribed cartridges and other subscription groups (nested grouping) ( by means of referencing their alias)
+ dependencies between components (startup, termination, etc)
+ inline definition of subscription_groups in json doc (see example further down)
Going with this proposal I think we could leave current cartridge subscription unchanged but tie the subscribed cartridges to a subscription plan. Startup and termination dependencies will be checked by the autoscaler.
Please see also the section “Subscription Plan”in the shared google doc
I would like to call it as "Composite Application" instead of "Subscription Plan".
What do you think ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
Json example:
{
"subscriptionPlanId": "subscriptionId",
"alias": "subscription name",
"components": [
{
"configGroup": {
"alias": "group1",
"subscribables": [
"c1",
"group2"
],
"dependencies": {
"startup_order": [
{
"key": "c1",
"value": "c2"
}
],
"kill_behavior": "asymmetrical"
}
}
},
{
"configGroup": {
"alias": "group2",
"subscribables": [
"c3"
]
}
}
]
}
From: Martin Eppel (meppel)
Sent: Saturday, May 17, 2014 2:44 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Absolutely,
We want to incorporate the feedback of all the experts but in the meantime we can get started based on what we know and then gradually enhance.
IMHO it would also make sense to start developing the feature on a separate branch (of RC3 tag ) and gradually improve the feature until we feel it is ready to be incorporated into the mainline, this would allow us to have something ready by the end of the month without destabilizing the master branch ?
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list and get feedback as well. We might need to come with some sequence diagrams as well. I was planning to do that after we have the call tomorrow. This is a crucial and a core change, plus a very useful feature to Stratos, so better to get feedback from the experts in the dev list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON definition for the group subscription. My intention is to re-use the Subscription model that we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss during the call, and decide how to proceed.
On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Melan
Attached is the code I started implementing:
1. Added REST API for application deployment
2. Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)
I also attached the sample JSON I used to test the REST API:
Some of the code is experimental, e.g publishing the ApplicationCreatedEvent from ServiceUtil with the purpose to test the messaging infrastructure and to be able to implement and test the backend changes (application json) in the autoscaler (not there yet).
I think there is still a bunch of things missing like an application deployment / subscriptions manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I was thinking you could focus on the deployment logic, messaging, etc … while I take a look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.
Regards
Martin
From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Isuru,
Just confirming the meeting time Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
Regards
Martin
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru,
What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)
How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.
I noticed that emails are getting very slowly processed on dev@stratos list so I am sending it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.
Btw, we are trying to get this feature ready (at least in some basic form) by the end of the month
I'm positive we can get an initial version working by the end of this month, if all goes well.
Thanks
Martin
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things on the proposal and the concepts you brought in, simplify it further if possible, and decide how to proceed.
On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, see inline
From: isuruh@wso2.com<ma...@wso2.com> [mailto:isuruh@wso2.com<ma...@wso2.com>] On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>; Lakmal Warusawithana; Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Isuru, Shaheed
Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw, was left out accidentally, thanks for pointing it out),
Ad single definition:
Makes sense too, however we should still consider that the json definition can become fairly complex when multiple components and sub components are defined, that’s the reason I think it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate them in the same application json document, but not like a “in-line” definition of each component?)
Exactly . This is fine as the first step IMHO.
Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer to the relevant policies (which are already deployed) in the service group definition. The service group should contain the group specific details IMHO, as the dependencies, startup orders, kill behaviours, etc. WDYT?
Martin: +1 (see my example json definitions)
We could provide eventually both options, in-line with all components defined in one document (simple configurations) or modular, where components are deployed (un-deployed) individually (for large, complex configurations).
In the shared document I provided a full example of the json definition of an application, and just adding a few components seemed to make the document fairly large or complex (see chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the next steps.
Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?
Thanks
Martin
https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
From: Isuru Haththotuwa [mailto:isuruh@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally to deploying component JSON definitions one by one, shall we allow to deploy a one single definition (that will have all the dependencies and other information defined) to be deployed? WDYT? IMHO this will be more user friendly.
On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <sh...@cisco.com>> wrote:
A couple of points on “kill behaviour”:
• I don’t think the terms “asymmetric” and “symmetric” capture the behaviours I had in mind. For me there were two use cases:
o Assuming A depends on B and B depends on C
o And B fails
o Use case #1 is that only A should be killed and allowed to restart. We could call this kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier architecture where A is the front end, B is the application logic and C is the database.
o Use case #2 is that both A and C should be killed and allowed to restart. We could call this “kill-all”. This fits the use case of a poorly behaved application where everything needs a restart on any failure.
• Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow convince ourselves this was not needed? I’m actually tending towards the latter since if kill-none were a usable value, it seems to contradict that there was a dependency in the first place.
+1.
Thanks, Shaheed
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: 07 May 2014 06:48
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
See inline "Martin:"
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this, we can start implementing step by step. will do this by milestone by milestone approach.
Further I think we can used same topology topic to create this model. With this root node of the topology will be application. From the auto scaler we can implement application monitor to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included
As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge definitions) existing definitions to define application with new requirement like grouping, dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies, deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents are referenced by name so each of these have to be deployed separately (as they are today) and when the application is deployed it will reference the deployed artifacts - what do you think ?
We'll might have to add some extra properties - I'll work on a complete example and post the progress on Google Doc.
Then we should divide each work item and start implementing. Ideally in the milestone one should not break existing, but reused them. Then later we can identifying what need to improve/change existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ? Also, I totally agree, all code change should be complementary without breaking any existing features
Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either today, tomorrow or then again next week Wednesday.
Seems like we have to wait till next Friday, next week, Wednesday and Thursday Holiday in Sri Lanka. Anyway will start working on this and will identify initial task and will post here.
Lets have a hangout at a convenient time to discuss the next steps.
On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Works for me, I’ll add the content
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
Thanks for detail description. We can create wiki space , but IMHO its easy if we can use public google doc. I have created [1] shall we add our proposal into it. May be we need some hangout session to discuss further on this.
[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
I summarized and formatted the discussion points from the previous email(s) thread for better readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request (JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there a wiki to post and discuss proposals like this ?
Thanks
Martin
Grouping of Cartridges / Services:
1. Application model
1.1 General model
See proposal below
1.2 Json
see proposal below
2. Application deployment
2.1 Deployment
+ application deployment (deploy)
+ application removal (undeploy)
2.2 Subscription
+ subscription (subscribe)
+ cancelation of subscription (unsubscribe)
2. Distributing Application deployment
2.1 Publishing application deployment
+ new topic to publish application events, map, status
2.2 Subscribing to published application deployment
+ Autoscaler
3. Dependency calculation
3.1 Building static dependency tree
3.1.1 Calculating dependency tree, dependency check in Autoscaler
+ Dependency tree determines all dependencies,
+ symmetrical / asymmetrical dependencies
+ Dependency checks member status for all dependencies
successful for at least one member in ACTIVE state, failed for no member in ACTIVE state
3.1.1.1 Enhancing autoscaler rules
+ Integrate with
mincheck.drl to control instance creation, termination rule to control instance termination
4. Application event model
+ provides state information for application, application components like
Up, Down, partial up
From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)
Hi Lakmal at al
Here is a first draft for an application model as a starting point, it builds on the previous discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.
Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component
+ description ... properties (like name, type)
+ components ... children, nested
+ configuration ... properties
+ dependencies ... puts immediate components (children) in relation
Generic Component format:
+ description (component specific properties)
+ components (by reference)
+ configuration
- described through properties
+ properties can be grouped, group can be referenced
+ dependencies
- described through properties
In json:
Generic Component:
Component:
{
"description": [
],
"configuration":[
],
"components":[
],
"dependencies":[
]
}
Application:
{
"description": [
{"name":"application name"},
{"types":["application", "subscribable"]}
],
"configuration":[
{"deployment":"deployment_policy_name"}
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical or asymmetrical"}
]
}
Subscribable (aka groups):
{
"description": [
{"name":"component name"},
{"types":["subscribable"]}
],
"configuration":[
],
"components":[
{"subscribables": ["subscribable name",
"subscribable name",
"subscribable name"]}
],
"dependencies":[
{"startup_order": [
{"dependant name": "subscribable name"},
{"dependant name": "subscribable name"}
]},
{"kill_behavior": "symmetrical / asymmetrical"}
]
}
Cartridge Component:
{
"description": [
{"name":"component name"},
{"types":["subscribable", "scalable"]}
],
"configuration":[
{"scaling":"autoscaling policy name"},
{"info":"cartridge info"}
],
"components":[
],
"dependencies":[
]
}
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com>> wrote:
Hi,
Sorry for top-posting, but one additional thing that I feel should be considered is a minimal set of lifecycle events for the group; for example:
- "all members of a group are now active", aka "group active".
- "at least one member of an existing group has failed", aka "group down".
This is clearly an add-on to the core functionality but is needed to round the feature out.
+1
Thanks, Shaheed
On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,
With the current implementation I think and agree its better to go with json as defining configuration data. Summarizing proposal and the discussion I can think following data model. Here I am thinking implementing single configuration data (application deployment definition) model, which can define full configuration of an composite application. we should take it down to several milestones. we can start on enhancing backend services in initial milestone and later will come with aggregating them. All of your thoughts welcome.
Can you come up with some definition format. (may be json format). Also see my inline comment.
On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions and questions) :
+ Defining configuration data format – for example json or chef / recipe
++ would using recipes affect all configuration data or is it restricted to the grouping / dependency configuration only ?
IMHO, its may easy with json, considering current existing services.
+ Adding the new model to stratos manager and integrate with existing model:
++ Group model (nested groups, group properties)
++ Dependency model
++ in the current model we have a Subscriptions and Deployment manager, I think we need something similar to handle the groups which in turn invokes deployment / subscriptions manager
+1
+ Defining runtime dependency conditions:
++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency tree and checking the state of dependencies (e.g. member state)
+1
+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies and act accordingly):
++ As of now I think it has to be integrated with the autoscaler as it the entity which controls instances (members) and checks their states
Yes, we need to enhance auto scaler
++ How is dependency retrieved from configuration data (e.g. published through topology, beans, etc ….) ?
We need to think on more this. IMO, we may need another topic (application deployment definition) to retrieved the config data. SM can define this based on provide definition, auto scaler and who ever need can get that. Topology is maintaining current running state (currently its for a service, we need to enhance it to application), so we can used "application deployment definition" data to check with current state.
++ Integrate dependency checks into the runtime portion of the autoscaler, e.g. add the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning / terminating instances)
I think we can coupled auto scaling policy with individual cartridges/services.
What do you think ?
We may need to think about deployment policy, I think with the application deployment definition (with some coupling ,like some cartridges may need to spin up same partition..etc). May be we are redefining deployment policy with application deployment definition. (we may obsolete deployment policy)
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to all cartridges alike most importantly it will allow us also to define dependencies between cartridges. Groups should also be flexible enough to not only contain single cartridges but also groups, allowing a hierarchical structure.
Dependencies are defined at the group level, describing the dependency relationship of immediate children.
Other group specific properties could be defined which will apply to all immediate group members.
The subscription model will be extended to subscribe to a group in addition to a cartridge (either or). To generalize we added the concept of a Subscribable which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics, it seemed appropriate to add the concept of Scalable, describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
* A Group can be subscribed or a Scalable can be subscribed, whereas
today, a Cartridge is subscribed.
o The new Scalable entity is needed because currently
Subscriptions own autoscale policies, but that doesn’t work when
a Group contains more than one Cartridge, because we want one
autoscale policy per Cartridge. However we can’t couple the
autoscale policy to the Cartridge because it may be re-used in
other Subscriptions where a different autoscale policy is desired
* children is the set of Subscribables in the Group.
o This supports hierarchical Groups.
* collocate says “these Children must be physically next to each other”
* startupOrder is a Set of asymmetric pairs (A, B) where A points to B
but B doesn’t point to A.
o These pairs define a DAG, which we use to perform a topological
sort of the children and get a /partial/ ordering.
o Any member of “children” not in a pair is a singleton started in
parallel to everything else.
* The killBehaviour says that when node X dies, we must kill:
o Everything else (for applications where the loss of any member
of “children” cannot be recovered with less impact
o Nothing else (for applications where any element can be restarted)
o Or just the things “upstream” in the startup order (for
conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of things like “n instances of A depends on B”; that can be considered in the future as an enhancement.
Startup use cases considered
* All start in parallel: don’t specify any pairs => everything is
equivalent => all start in parallel
* All start one-after-another (total order): specify a set of pairs
such that there is a single contiguous list covering all the nodes,
e.g. A -> B -> C -> D -> E
* Some start-up dependencies: specify as necessary e.g. [[MiddleTier
-> Database], [Logging]]
Kill use cases considered
* One dies => all die: set killBehaviour to KillAll
* One dies => nothing happens: set killBehaviour to killNone
* One dies => its dependancies die (aka “restart from here”): specify
a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
-> Database], [Logging]], Database dies => MiddleTier is restarted,
Logging untouched.
Example instantiation
Description: see attachment Slide2.png
Thanks
Martin
From: Lakmal Warusawithana [mailto:lakmal@wso2.com<ma...@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).
However, I am not sure what the alternative might be to solve the requirements (e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing, dependent scaling and dependent termination to the autoscaler defined by optional properties in the autoscaler policy.
From my current point of view, CAMP seems a fairly complex specification which might require quite some changes to adopt.
Sorry I could not go through CAMP in detail. Should spend sometime.
I do agree that alternatives might exist and should be discussed !? Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we can match with existing Stratos workflow IMO.
Thanks
Martin
From: damitha kumarage [mailto:damitha23@gmail.com<ma...@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<ma...@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
I think this property will be a quite useful piece to solve the grouping issue:
I also would like to suggest to add the serviceGroup to the topology map (in case it is not yet available in the topology map). This will help to tie together cartridges (or services) in the autoscaler and , for example enable synchronized auto scaling behavior of services within a service group, like synchronized scaling, sequenced boot up, etc ….
In addition the autoscaler should be enhanced to add additional (but optional properties) in the auto scaling policy related to a service group to govern the respective auto scaling behavior.
For example, related properties should identify a service group and other related properties to define dependencies between the various cartridges in the service group like boot sequence, scale up / down ratios, termination dependencies, etc … . The property set (or json structure ) should be fairly flexible as we are just about to explore this new feature and should be easily expandable.
I would think that these additions will also prove useful to integrate in the long term with CAMP (or other spec) but will help to solve more immediate requirements
Yes, these are very valid points in coming up with a proper grouping architecture for services. Thank you for bringing them up.
As I understand, what Sajith has done here is enabling static grouping of services, by using a property for that in the cartridge deployment time. What we are trying to achieve in long term is dynamic grouping of services, so that we can group any available service at runtime seamlessly, according to CAMP specification (or some other suitable way).
I see three things here
1) Grouping and resolve inter-dependancies between cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and inter-dependancy( of cartridges. There can also be intra-dependancies inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are compelled to adhere). Is there a way we can solve this without doing major changes to existing components? For example without complicating autoscaler policies/logic as suggested by Martin above?
Damitha
--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Thanks and Regards,
Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: adding the groupId to a member event ... RE: [Discuss] Grouping
of services (cartridges)
Posted by Nirmal Fernando <ni...@gmail.com>.
Hi Martin,
Who knows about this group id? Who's suppose to set it to the topology?
On Wed, Jun 4, 2014 at 2:21 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Hi Isuru,
>
>
>
> As part of the grouping I need to add the groupId to a Member event (e.g.
> MemberStartedEvent, MemberActivatedEvent).
>
> Currently I added the composite application data to the Topology by
> generating a event and respective event processor (similar to Service)
> which works fine for the autoscaler.
>
> However it seems that the CloudController uses its own registry and the
> application info is missing when the topology is restored so it seems I
> have to push the application data also to the cloud controller.
>
> The only interface available seems to be the soap api
> (CloudControllerService.wsdl) which I would like to avoid (complicated and
> time intensive). Is there a better way to push the data, can I use a
> topology event, share the manager registry or do you have any other
> recommendations ?
>
> In case the soap API is the only option, is there a better way to
> generate the CloudControllerService .wsdl or do I have to manually edit in
> my API changes ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Tuesday, May 20, 2014 9:15 AM
> *To:* Isuru Haththotuwa
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sounds good
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Tuesday, May 20, 2014 4:41 AM
> *To:* Martin Eppel (meppel)
> *Cc:* Lakmal Warusawithana; dev@stratos.incubator.apache.org; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Tue, May 20, 2014 at 12:17 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Can you give me a quick json example to make sure I won’t misunderstand ?
>
> We would still be using the component JSON definitions as per your
> proposal, since if we put everything inline it would be one large JSON
> file. In the top level Composite Application Definition (which is the more
> appropriate name for subscription Plan, as suggested by Lakmal), you can
> refer the previously deployed groups by using the alias. So, the last
> example that you have given in the doc would be a valid JSON sample. Hope
> this is clear. Do let me know if its not.
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 11:33 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 11:55 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Lakmal,
>
>
>
> A few clarifications:
>
>
>
> I think based on the last proposal we can consolidate the definition of
> components, dependencies in one definition and deploy all at once as
> subscription plan, correct (simplifies the deployment) ?
>
>
>
>
>
> Yes, but as the first step, will have separate deployers for groups,
> components and then composite application (which is you have mention
> subscription plan).
>
>
>
> Also, Isuru pointed out that the term application might be confusing so
> I suggested subscription plan (or something similar), do we still want to
> go with it ?
>
>
>
>
>
> What i suggest to call it composite application.
>
>
>
> And composite application can refer, previous deployed group, components.
> components can refer cartridge definitions. And dependancies (start order,
> kill order) should be defined in the top level, which is in composite
> application definition.
>
>
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:58 AM
> *To:* Martin Eppel (meppel); Lakmal Warusawithana
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Sorry, hit the send button to soon:
>
>
>
> Ok, I’ll work on the deployment. I’ll send you guys the code as patch to
> checkin.
>
>
>
> Do we have already a planned date for 4.1 Milestone 1 ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Monday, May 19, 2014 10:52 AM
> *To:* 'Lakmal Warusawithana'
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 9:45 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Mon, May 19, 2014 at 10:01 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
>
>
> This sounds good to me, how do you suggest to coordinate the development.
> Will it make sense to create a branch (based on RC3 ?) to develop on ?
>
>
>
>
>
> No, lets work on master branch. Which part do you like to work on? Can you
> start work on deploy these new definitions?
>
>
>
> Need to have persist model also. IsuruH, can you guys come up with it.
>
>
>
> Melan, can you start extent Composite Application create event?
>
>
>
> Lahiru can you start working on Composite Application monitor?
>
>
>
> Will target 4.1.0 Milestone 1.
>
>
>
> Is this look like a plan :)
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* Monday, May 19, 2014 2:06 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Isuru Haththotuwa; Melan
> Jayasingha; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Shall we start implementation, I can see following tasks.
>
> - Extending SM API for
>
>
> - deploy components
> - deploy group
> - deploy composite application definition
>
>
> - persist above deployment
> - validation
> - Extend SM for Composite Application creation in the topology
> (earlier we had cluster create)
> - Extend Auto Scaler (implementing Composite Application monitor)
>
>
>
> On Mon, May 19, 2014 at 2:15 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
> Thanks Martin, get clear idea now.
>
>
>
> On Sun, May 18, 2014 at 10:45 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Based on our last night conversation I want to slightly modify the
> previous grouping proposal:
>
>
>
> Instead of defining an application (misleading naming) we define a
> subscription plan:
>
>
>
> The subscription defines the following:
>
> + grouping of components (or subscription_groups)
>
> + subscription_groups logically group subscribed cartridges and other
> subscription groups (nested grouping) ( by means of referencing their alias)
>
> + dependencies between components (startup, termination, etc)
>
> + inline definition of subscription_groups in json doc (see example
> further down)
>
>
>
> Going with this proposal I think we could leave current cartridge
> subscription unchanged but tie the subscribed cartridges to a subscription
> plan. Startup and termination dependencies will be checked by the
> autoscaler.
>
>
>
> Please see also the section “*Subscription Plan”*in the shared google doc
>
>
>
>
>
> I would like to call it as "Composite Application" instead of
> "Subscription Plan".
>
>
>
>
>
> What do you think ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.hkmqg1beblld
>
>
>
>
>
> *Json example:*
>
>
>
> {
>
> "subscriptionPlanId": "subscriptionId",
>
> "alias": "subscription name",
>
> "components": [
>
> {
>
> "configGroup": {
>
> "alias": "group1",
>
> "subscribables": [
>
> "c1",
>
> "group2"
>
> ],
>
> "dependencies": {
>
> "startup_order": [
>
> {
>
> "key": "c1",
>
> "value": "c2"
>
> }
>
> ],
>
> "kill_behavior": "asymmetrical"
>
> }
>
> }
>
> },
>
> {
>
> "configGroup": {
>
> "alias": "group2",
>
> "subscribables": [
>
> "c3"
>
> ]
>
> }
>
> }
>
> ]
>
> }
>
>
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Saturday, May 17, 2014 2:44 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Absolutely,
>
>
>
> We want to incorporate the feedback of all the experts but in the meantime
> we can get started based on what we know and then gradually enhance.
>
>
>
> IMHO it would also make sense to start developing the feature on a
> separate branch (of RC3 tag ) and gradually improve the feature until we
> feel it is ready to be incorporated into the mainline, this would allow us
> to have something ready by the end of the month without destabilizing the
> master branch ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Saturday, May 17, 2014 10:16 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal
> Warusawithana; Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> Thanks a lot for your efforts. Will try to have a look at the code before
> the call.
>
> IMHO, before adding our changes, it is better to share the intended design
> with the dev list and get feedback as well. We might need to come with some
> sequence diagrams as well. I was planning to do that after we have the call
> tomorrow. This is a crucial and a core change, plus a very useful feature
> to Stratos, so better to get feedback from the experts in the dev list.
> WDYT?
>
> I prepared some JSON definition referring your ones, in which I have only
> one single JSON definition for the group subscription. My intention is to
> re-use the Subscription model that we have in Stratos Manager and extend it
> to support groups and dependencies. Lets discuss during the call, and
> decide how to proceed.
>
>
>
> On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Melan
>
>
>
> Attached is the code I started implementing:
>
> 1. Added REST API for application deployment
>
> 2. Added ApplicationCreatedEvent (+ some corresponding messaging
> infrastructure)
>
>
>
> I also attached the sample JSON I used to test the REST API:
>
>
>
> Some of the code is experimental, e.g publishing the
> ApplicationCreatedEvent from ServiceUtil with the purpose to test the
> messaging infrastructure and to be able to implement and test the backend
> changes (application json) in the autoscaler (not there yet).
>
>
>
> I think there is still a bunch of things missing like an application
> deployment / subscriptions manager (similar to deploying a cartridge, etc
> …), but I am not entirely sure what you guys have in mind there.
>
> I do have some idea what should be done in the autoscaler to add
> dependencies check, so I was thinking you could focus on the deployment
> logic, messaging, etc … while I take a look at the autoscaler stuff.
>
> Anyway, this is just a suggestion, take a look and we can discuss more
> details in our call.
>
>
>
> Regards
>
>
>
>
>
> Martin
>
>
>
> *From:* Martin Eppel (meppel)
> *Sent:* Friday, May 16, 2014 8:16 PM
> *To:* Isuru Haththotuwa
>
>
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
>
> *Subject:* RE: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Isuru,
>
>
>
> Just confirming the meeting time Saturday 8.30 PST is ok with me (My
> time would be Sunday 8.30 AM IST)
>
>
>
> Regards
>
>
>
> Martin
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Friday, May 16, 2014 10:50 AM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu); Melan Jayasingha
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
> On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru,
>
>
>
> What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if
> this doesn’t work we can meet up Sun 8:00 pm PST (8:30 am IST ?)
>
> Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM
> IST)
>
>
>
> How will we connect, I haven’t used Google hangout yet ?
>
> We can connect through either Google hangout/ Skype, whatever easier for
> you.
>
>
>
> I noticed that emails are getting very slowly processed on dev@stratos
> list so I am sending it also directly to you as well,
>
> I too noticed that your last mail got delayed by approx. one day.
>
>
>
> Btw, we are trying to get this feature ready (at least in some basic form)
> by the end of the month
>
> I'm positive we can get an initial version working by the end of this
> month, if all goes well.
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Thursday, May 15, 2014 9:10 PM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
> I'm in IST. Please suggest a convenient time for you. I would like to
> clarify several things on the proposal and the concepts you brought in,
> simplify it further if possible, and decide how to proceed.
>
>
>
> On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, see inline
>
>
>
> *From:* isuruh@wso2.com [mailto:isuruh@wso2.com] *On Behalf Of *Isuru
> Haththotuwa
> *Sent:* Wednesday, May 14, 2014 1:12 PM
> *To:* Martin Eppel (meppel)
> *Cc:* dev@stratos.incubator.apache.org; Lakmal Warusawithana; Shaheedur
> Haque (shahhaqu)
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin,
>
>
>
> On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Isuru, Shaheed
>
>
>
>
>
> Ad “kill behaviour”:
> +1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none”
> (btw, was left out accidentally, thanks for pointing it out),
>
>
>
> Ad single definition:
>
>
>
> Makes sense too, however we should still consider that the json
> definition can become fairly complex when multiple components and sub
> components are defined, that’s the reason I think it would still make sense
> to keep it modular, at least as an option.
>
> +1
>
> Martin: +1
>
> (btw, I assume you suggest instead of defining each component in
> separate files we aggregate them in the same application json document,
> but not like a “in-line” definition of each component?)
>
> Exactly . This is fine as the first step IMHO.
>
>
>
> Martin: +1 ( inline wouldn’t be a good choice IMHO)
>
> Does this include other definitions like cartridge info definition,
> deployment policy, autoscaling definition, etc ... ?
>
> No. I agree that if we do this, it will be an unmanageably large JSON
> definition. We can refer to the relevant policies (which are already
> deployed) in the service group definition. The service group should contain
> the group specific details IMHO, as the dependencies, startup orders, kill
> behaviours, etc. WDYT?
>
>
>
> Martin: +1 (see my example json definitions)
>
>
>
> We could provide eventually both options, in-line with all components
> defined in one document (simple configurations) or modular, where
> components are deployed (un-deployed) individually (for large, complex
> configurations).
>
>
>
> In the shared document I provided a full example of the json definition of
> an application, and just adding a few components seemed to make the
> document fairly large or complex (see chapter “Json example”).
>
> Great! Thanks Martin. Lets do a hangout, preferably on Friday so that
> we can decide on the next steps.
>
>
>
> Martin: sounds good, Friday would be Thursday my time, which time zone are
> you in ?
>
>
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p
>
>
>
>
>
> *From:* Isuru Haththotuwa [mailto:isuruh@wso2.com]
> *Sent:* Tuesday, May 13, 2014 6:08 AM
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Lakmal Warusawithana
>
>
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
> Hi Martin, Shaheedur and all,
>
> Went through the shared document for the proposal. good work! One small
> suggestion, additionally to deploying component JSON definitions one by
> one, shall we allow to deploy a one single definition (that will have all
> the dependencies and other information defined) to be deployed? WDYT? IMHO
> this will be more user friendly.
>
>
>
>
>
> On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <
> shahhaqu@cisco.com> wrote:
>
> A couple of points on “kill behaviour”:
>
>
>
> · I don’t think the terms “asymmetric” and “symmetric” capture
> the behaviours I had in mind. For me there were two use cases:
>
> o Assuming A depends on B and B depends on C
>
> o And B fails
>
> o Use case #1 is that only A should be killed and allowed to restart.
> We could call this kill behaviour “kill-dependents” since A is dependent on
> B. This fits the standard n-tier architecture where A is the front end, B
> is the application logic and C is the database.
>
> o Use case #2 is that both A and C should be killed and allowed to
> restart. We could call this “kill-all”. This fits the use case of a poorly
> behaved application where everything needs a restart on any failure.
>
> · Did we accidentally drop the third value, i.e. “kill-none”, or
> did we somehow convince ourselves this was not needed? I’m actually tending
> towards the latter since if kill-none were a usable value, it seems to
> contradict that there was a dependency in the first place.
>
> +1.
>
>
>
> Thanks, Shaheed
>
>
>
> *From:* Lakmal Warusawithana [mailto:lakmal@wso2.com]
> *Sent:* 07 May 2014 06:48
>
>
> *To:* dev@stratos.incubator.apache.org
> *Cc:* Shaheedur Haque (shahhaqu)
> *Subject:* Re: [Discuss] Grouping of services (cartridges)
>
>
>
>
>
>
>
> On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi Lakmal,
>
> See inline "Martin:"
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
>
> Sent: Tuesday, May 06, 2014 3:25 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for adding content, I had offline discussion with Imesh, Lahiru and
> IsuruH on this, we can start implementing step by step. will do this by
> milestone by milestone approach.
>
> Further I think we can used same topology topic to create this model. With
> this root node of the topology will be application. From the auto scaler we
> can implement application monitor to handle this. service clusters will be
> sub set of application.
>
> Martin: +1, dependencies have to be included
>
>
> As the first step we reused (partitions, auto scaling policy, deployment
> policy, cartridge definitions) existing definitions to define application
> with new requirement like grouping, dependency, start order ..etc. Shall we
> create sample json ( with sample auto scaling policies, deployment
> policies, cartridges ..etc) to get some idea for the complexity? (still I
> am figuring out some definition like subscribable and some real sample json)
>
> Martin: +1, Actually, the json example I posted on Google docs assumes
> that we'll be reusing all the existing artifacts (like cartridge definition
> etc ...). Cartridges, policies, componenents are referenced by name so each
> of these have to be deployed separately (as they are today) and when the
> application is deployed it will reference the deployed artifacts - what do
> you think ?
> We'll might have to add some extra properties - I'll work on a complete
> example and post the progress on Google Doc.
>
>
>
> Then we should divide each work item and start implementing. Ideally in
> the milestone one should not break existing, but reused them. Then later we
> can identifying what need to improve/change existing components.
>
> Martin: +1, would it speed up / simply implementation if we start working
> on a branch ? Also, I totally agree, all code change should be
> complementary without breaking any existing features
>
>
> Later this week we can have google hangout session to discuss further on
> this.
>
> Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can
> do a hangout either today, tomorrow or then again next week Wednesday.
>
>
>
> Seems like we have to wait till next Friday, next week, Wednesday and
> Thursday Holiday in Sri Lanka. Anyway will start working on this and will
> identify initial task and will post here.
>
>
> Lets have a hangout at a convenient time to discuss the next steps.
>
>
> On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Works for me, I’ll add the content
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, May 04, 2014 8:56 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> Thanks for detail description. We can create wiki space , but IMHO its
> easy if we can use public google doc. I have created [1] shall we add our
> proposal into it. May be we need some hangout session to discuss further on
> this.
>
> [1]
> https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit
>
>
> On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> I summarized and formatted the discussion points from the previous
> email(s) thread for better readability and reference.
> Btw, what is the appropriate way going forward to elevate the discussion
> to a feature request (JIRA) ? Also, how do we typically discuss feature
> requests like this, the email format makes it somewhat difficult to read
> and it is easy to overlook posted discussion points ? Is there a wiki to
> post and discuss proposals like this ?
>
> Thanks
>
> Martin
>
> Grouping of Cartridges / Services:
>
> 1. Application model
> 1.1 General model
> See proposal below
> 1.2 Json
> see proposal below
>
> 2. Application deployment
> 2.1 Deployment
> + application deployment (deploy)
> + application removal (undeploy)
> 2.2 Subscription
> + subscription (subscribe)
> + cancelation of subscription (unsubscribe)
>
> 2. Distributing Application deployment
> 2.1 Publishing application deployment
> + new topic to publish application events, map, status
> 2.2 Subscribing to published application deployment
> + Autoscaler
>
> 3. Dependency calculation
> 3.1 Building static dependency tree
> 3.1.1 Calculating dependency tree, dependency check in Autoscaler
> + Dependency tree determines all dependencies,
> + symmetrical / asymmetrical dependencies
> + Dependency checks member status for all dependencies
> successful for at least one member in ACTIVE state, failed
> for no member in ACTIVE state
> 3.1.1.1 Enhancing autoscaler rules
> + Integrate with
> mincheck.drl to control instance creation,
> termination rule to control instance termination
>
>
> 4. Application event model
> + provides state information for application, application components
> like
> Up, Down, partial up
>
>
>
> From: Martin Eppel (meppel)
> Sent: Thursday, May 01, 2014 3:26 PM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: RE: [Discuss] Grouping of services (cartridges)
>
> Hi Lakmal at al
>
> Here is a first draft for an application model as a starting point, it
> builds on the previous discussions (and probably leaves out a few things):
> Also, to add to Shaheeds point there is probably an event model around
> this as well.
>
> Definitions:
> + composite structure
> + A component is sectioned into description, components, configuration,
> dependencies.
> + Components are typed (application, subscribables, scalable)
> + Application is also component
>
> + description ... properties (like name, type)
> + components ... children, nested
> + configuration ... properties
> + dependencies ... puts immediate components (children) in relation
>
> Generic Component format:
> + description (component specific properties)
> + components (by reference)
> + configuration
> - described through properties
> + properties can be grouped, group can be referenced
> + dependencies
> - described through properties
>
>
> In json:
>
> Generic Component:
>
> Component:
>
> {
> "description": [
> ],
> "configuration":[
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Application:
>
> {
> "description": [
> {"name":"application name"},
> {"types":["application", "subscribable"]}
> ],
> "configuration":[
> {"deployment":"deployment_policy_name"}
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical or asymmetrical"}
> ]
> }
>
> Subscribable (aka groups):
>
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable"]}
> ],
> "configuration":[
> ],
> "components":[
> {"subscribables": ["subscribable name",
> "subscribable name",
> "subscribable name"]}
> ],
> "dependencies":[
> {"startup_order": [
> {"dependant name": "subscribable name"},
> {"dependant name": "subscribable name"}
> ]},
> {"kill_behavior": "symmetrical / asymmetrical"}
> ]
> }
>
> Cartridge Component:
> {
> "description": [
> {"name":"component name"},
> {"types":["subscribable", "scalable"]}
> ],
> "configuration":[
> {"scaling":"autoscaling policy name"},
> {"info":"cartridge info"}
> ],
> "components":[
> ],
> "dependencies":[
> ]
> }
>
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Thursday, May 01, 2014 1:18 AM
> To: Shaheedur Haque (shahhaqu)
> Cc: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <sh...@cisco.com> wrote:
> Hi,
>
> Sorry for top-posting, but one additional thing that I feel should be
> considered is a minimal set of lifecycle events for the group; for example:
>
> - "all members of a group are now active", aka "group active".
>
> - "at least one member of an existing group has failed", aka "group down".
>
> This is clearly an add-on to the core functionality but is needed to round
> the feature out.
>
>
> +1
>
> Thanks, Shaheed
>
>
> On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
> Hi Martin,
>
> With the current implementation I think and agree its better to go with
> json as defining configuration data. Summarizing proposal and the
> discussion I can think following data model. Here I am thinking
> implementing single configuration data (application deployment definition)
> model, which can define full configuration of an composite application. we
> should take it down to several milestones. we can start on enhancing
> backend services in initial milestone and later will come with aggregating
> them. All of your thoughts welcome.
>
> Can you come up with some definition format. (may be json format). Also
> see my inline comment.
>
> On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal,
>
> Below are ideas on how to break up the feature into a high level ToDo list
> (with some suggestions and questions) :
>
> + Defining configuration data format – for example json or chef / recipe
> ++ would using recipes affect all configuration data or is it
> restricted to the grouping / dependency configuration only ?
>
> IMHO, its may easy with json, considering current existing services.
>
>
> + Adding the new model to stratos manager and integrate with existing
> model:
> ++ Group model (nested groups, group properties)
> ++ Dependency model
> ++ in the current model we have a Subscriptions and Deployment
> manager, I think we need something similar to handle the groups which in
> turn invokes deployment / subscriptions manager
>
>
> +1
>
> + Defining runtime dependency conditions:
> ++ Calculating the dependency for a cartridge (service) dynamically
> by walking the dependency tree and checking the state of dependencies (e.g.
> member state)
>
>
> +1
>
> + Integrate the dynamic aspect of the dependency model (checking the
> conditions of dependencies and act accordingly):
> ++ As of now I think it has to be integrated with the autoscaler as
> it the entity which controls instances (members) and checks their states
>
> Yes, we need to enhance auto scaler
>
> ++ How is dependency retrieved from configuration data (e.g.
> published through topology, beans, etc ….) ?
>
> We need to think on more this. IMO, we may need another topic (application
> deployment definition) to retrieved the config data. SM can define this
> based on provide definition, auto scaler and who ever need can get that.
> Topology is maintaining current running state (currently its for a service,
> we need to enhance it to application), so we can used "application
> deployment definition" data to check with current state.
>
> ++ Integrate dependency checks into the runtime portion of the
> autoscaler, e.g. add the checks to the monitor functionality of a
> ClusterMonitor with respective actions (spawning / terminating instances)
>
>
> I think we can coupled auto scaling policy with individual
> cartridges/services.
>
> What do you think ?
>
>
> We may need to think about deployment policy, I think with the application
> deployment definition (with some coupling ,like some cartridges may need to
> spin up same partition..etc). May be we are redefining deployment policy
> with application deployment definition. (we may obsolete deployment policy)
>
> Thanks
>
> Martin
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Wednesday, April 30, 2014 4:30 AM
>
> To: dev@stratos.incubator.apache.org
> Cc: Shaheedur Haque (shahhaqu)
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Hi Martin,
>
> These use cases are very valid, and we should integrate them into 4.1.0.
> Will go through in detail and see how we can incorporate into Stratos.
> I have one question, why do we need hierarchical grouping, (I mean group
> inside a group) and use case? Can't we have flat?
>
> On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi Lakmal
>
> Below is a further enhanced proposal to add grouping to apache stratos:
>
> In a nutshell:
>
> By grouping cartridges together we can define characteristics which apply
> to all cartridges alike most importantly it will allow us also to define
> dependencies between cartridges. Groups should also be flexible enough to
> not only contain single cartridges but also groups, allowing a hierarchical
> structure.
>
> Dependencies are defined at the group level, describing the dependency
> relationship of immediate children.
> Other group specific properties could be defined which will apply to all
> immediate group members.
>
> The subscription model will be extended to subscribe to a group in
> addition to a cartridge (either or). To generalize we added the concept of
> a Subscribable which can be either a group or cartridge.
>
> Since a Subscribale (or more specifically a group) doesn’t have scaling
> characteristics, it seemed appropriate to add the concept of Scalable,
> describing the scalable nature of a cartridge.
>
> Below is a more detailed description and a corresponding diagram:
>
> “Class diagram” of the logical model showing the existing description
> files in the Diadem model, and the proposed changes.
>
> Description: see attachment Slide1.png
>
> Some notes on the new dependency system:
>
> * A Group can be subscribed or a Scalable can be subscribed, whereas
> today, a Cartridge is subscribed.
> o The new Scalable entity is needed because currently
> Subscriptions own autoscale policies, but that doesn’t work when
> a Group contains more than one Cartridge, because we want one
> autoscale policy per Cartridge. However we can’t couple the
> autoscale policy to the Cartridge because it may be re-used in
> other Subscriptions where a different autoscale policy is desired
> * children is the set of Subscribables in the Group.
> o This supports hierarchical Groups.
> * collocate says “these Children must be physically next to each other”
> * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
> but B doesn’t point to A.
> o These pairs define a DAG, which we use to perform a topological
> sort of the children and get a /partial/ ordering.
> o Any member of “children” not in a pair is a singleton started in
> parallel to everything else.
> * The killBehaviour says that when node X dies, we must kill:
> o Everything else (for applications where the loss of any member
> of “children” cannot be recovered with less impact
> o Nothing else (for applications where any element can be restarted)
> o Or just the things “upstream” in the startup order (for
> conventional tiered applications)
>
> We think this describes all the use-cases we’ll ever see with the
> exception of things like “n instances of A depends on B”; that can be
> considered in the future as an enhancement.
>
> Startup use cases considered
>
> * All start in parallel: don’t specify any pairs => everything is
> equivalent => all start in parallel
> * All start one-after-another (total order): specify a set of pairs
> such that there is a single contiguous list covering all the nodes,
> e.g. A -> B -> C -> D -> E
> * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
> -> Database], [Logging]]
>
> Kill use cases considered
>
> * One dies => all die: set killBehaviour to KillAll
> * One dies => nothing happens: set killBehaviour to killNone
> * One dies => its dependancies die (aka “restart from here”): specify
> a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
> -> Database], [Logging]], Database dies => MiddleTier is restarted,
> Logging untouched.
>
> Example instantiation
>
> Description: see attachment Slide2.png
>
> Thanks
>
> Martin
>
>
> From: Lakmal Warusawithana [mailto:lakmal@wso2.com]
> Sent: Sunday, March 30, 2014 9:09 PM
>
> To: dev@stratos.incubator.apache.org
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
>
>
> On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> I totally agree, I am also not in favor of complicating existing
> components (e.g. autoscaler).
>
> However, I am not sure what the alternative might be to solve the
> requirements (e.g. item 1) mentioned below.
> The suggestion I made to enhance the autoscaler / rules are based on my
> understanding that the autoscaler already handles similar actions (e.g. VM
> startup, scaling and termination).
> It seems to me a logical extension to add additional knowledge to handle
> the boot sequencing, dependent scaling and dependent termination to the
> autoscaler defined by optional properties in the autoscaler policy.
>
> From my current point of view, CAMP seems a fairly complex specification
> which might require quite some changes to adopt.
>
> Sorry I could not go through CAMP in detail. Should spend sometime.
>
>
> I do agree that alternatives might exist and should be discussed !?
> Alternatives ?
>
> +1 for alternatives, we should not take CAMP as it is, we should see how
> we can match with existing Stratos workflow IMO.
>
>
> Thanks
>
> Martin
>
> From: damitha kumarage [mailto:damitha23@gmail.com]
> Sent: Sunday, March 30, 2014 7:59 AM
> To: dev@stratos.incubator.apache.org
>
> Subject: Re: [Discuss] Grouping of services (cartridges)
>
> Please see my inline comment,
>
> On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <is...@wso2.com>
> wrote:
> Hi Martin,
> On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
> Hi,
>
> I think this property will be a quite useful piece to solve the grouping
> issue:
>
> I also would like to suggest to add the serviceGroup to the topology map
> (in case it is not yet available in the topology map). This will help to
> tie together cartridges (or services) in the autoscaler and , for example
> enable synchronized auto scaling behavior of services within a service
> group, like synchronized scaling, sequenced boot up, etc ….
>
> In addition the autoscaler should be enhanced to add additional (but
> optional properties) in the auto scaling policy related to a service group
> to govern the respective auto scaling behavior.
>
> For example, related properties should identify a service group and other
> related properties to define dependencies between the various cartridges in
> the service group like boot sequence, scale up / down ratios, termination
> dependencies, etc … . The property set (or json structure ) should be
> fairly flexible as we are just about to explore this new feature and should
> be easily expandable.
>
> I would think that these additions will also prove useful to integrate in
> the long term with CAMP (or other spec) but will help to solve more
> immediate requirements
>
>
> Yes, these are very valid points in coming up with a proper grouping
> architecture for services. Thank you for bringing them up.
>
> As I understand, what Sajith has done here is enabling static grouping of
> services, by using a property for that in the cartridge deployment time.
> What we are trying to achieve in long term is dynamic grouping of services,
> so that we can group any available service at runtime seamlessly, according
> to CAMP specification (or some other suitable way).
>
> I see three things here
> 1) Grouping and resolve inter-dependancies between cartridges.
> 2) Composite Artifact deployment (May be if required, according to the
> above grouping and inter-dependancy( of cartridges. There can also be
> intra-dependancies inside a cartridge in case of artifact deployment)
> 3) Improved nonitoring and health check of above intricacies.
> Instead of adopt CAMP to solve above, I think it is better to discuss and
> find ways that fits most naturally to the existing Stratos
> architecture(Unless CAMP is widely adopted and we are compelled to
> adhere). Is there a way we can solve this without doing major changes to
> existing components? For example without complicating autoscaler
> policies/logic as suggested by Martin above?
> Damitha
>
>
>
>
> --
> __________________________________________________________________
> Damitha Kumarage
> http://people.apache.org/
> __________________________________________________________________
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Isuru H.
>
> +94 716 358 048
>
>
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
>
>
> --
>
> Lakmal Warusawithana
>
> Director - Cloud Architecture; WSO2 Inc.
>
> Mobile : +94714289692
>
> Blog : http://lakmalsview.blogspot.com/
>
>
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/