You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Lakmal Warusawithana <la...@wso2.com> on 2013/07/02 05:54:47 UTC
Re: Stratos Milestone plan
Hi all,
I will start separate threads to discuss each item in detail. What I can
see milestone one should be, changing code to apache and re-factor the code
to new structure. After discuss in detail will prioritize and then finalize
the next milestone items.
Current Stratos we use osgi model and will continue it. We will simplify
folder structure as below.
/
-Components
-- all osgi components
- Features (logically bundle components together)
-- adc
-- load balancer
-- auto scaller
-- manager
-- cloud controller
-- cli
-- cartridge agent
- Products (bundle several features and make a product)
- Service-stubs
-- all service stubs
- Tools
-- tools for create cartridges
Please share your thoughts.
thanks
On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
dkasunw@gmail.com> wrote:
> +1 for idea of load balance within static members. It allows to have more
> openings and be more flexible.
>
>
> Thank You,
> Dharshana.
>
>
> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <sa...@gmail.com>wrote:
>
>>
>>
>>
>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <nirmal070125@gmail.com
>> > wrote:
>>
>>> Hi Sanjeewa,
>>>
>>>
>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>> sanjeewa190@gmail.com> wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <la...@wso2.com>wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I hope all commiters / folks are in the mailing list now. Here I have
>>>>> listed below, some identified improvements, features that like to have in
>>>>> Stratos.
>>>>>
>>>>> Any other improvements/features are mostly welcome and will add those
>>>>> to the bellow list, then we can discuss and then will create milestones
>>>>> plan for releases. ( We will commit current code based soon after we got
>>>>> the git repository available)
>>>>>
>>>>> Before that if some one want to get quick overview of current
>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>> more detail [2] with current product documentation, which will migrate to
>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>> architecture and code based, and it may helpful other commierts to come
>>>>> up-to speed quickly.
>>>>>
>>>>> Here the list;
>>>>>
>>>>> - Re-factor current code to Apache
>>>>>
>>>>> First we will move current code based to apache and then start
>>>>> re-factoring to apache and make a release without adding any other
>>>>> improvement or feature.
>>>>>
>>>>>
>>>>> - IaaS Independent cartridge for Linux.
>>>>>
>>>>> With the current architecture when you create a cartridge its depend
>>>>> on the IaaS. For example If you create a cartridge in EC2, we have to make
>>>>> an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>> start separate mail thread with more detail on this)
>>>>>
>>>>>
>>>>> - Support Windows based cartridges.
>>>>>
>>>>> If IaaS support windows VM then we can have windows based cartridges.
>>>>> It will lead to have .net cartridges
>>>>>
>>>>>
>>>>> - Expand auto-scaling algorithm
>>>>>
>>>>> Currently auto scaling decision take based on looking at length of the
>>>>> http/https request queue. We have to expand this to consider CPU usage,
>>>>> memory utilization ..etc of the cartridges.
>>>>>
>>>>> Before we introduce que length based decision making mechanism we had
>>>> load average based decision making mechanism. Each server need to expose
>>>> some API(some service) to get load average of given server. Load
>>>> calculation can be different from one server to other. Load balancer will
>>>> do web service call to those end points and update load average values of
>>>> member list periodically.
>>>>
>>>
>>> We will separate out the load balancer logic and the auto-scaling logic
>>> soon. This would require some code level changes. But the vision is there.
>>>
>>> In brief, we should build APIs in Auto-scaler, so that the external
>>> components could provide/insert information that matters the decision of
>>> scaling up/down, depending on the algorithm used.
>>>
>>> Normally that service returns integer value(let say 0 to 10) and we
>>>> can get some idea about load from that. When it comes to traffic route we
>>>> will give high priority to server with minimum load and so on. So that type
>>>> of implementation would be ideal for this case as well. Synapse member
>>>> selection algorithm can be configurable and we can add new algorithm for
>>>> this if need.
>>>>
>>>
>>> This is an extension point, one could leverage.
>>>
>>>>
>>>>> - TCP level load balance.
>>>>>
>>>>> Current we only doing http/https based load balancing but need to
>>>>> expand this to do TCP level load balancing.
>>>>>
>>>>> This would be again nice feature IMO. Also i think we need
>>>> considerable amount of synapse transport level improvements to implement
>>>> this. WDYT?
>>>>
>>>
>>> We may require to write a TCP load balance endpoint, if one is not
>>> already available (I couldn't find any reference).
>>>
>>>>
>>>>> - Health monitoring
>>>>>
>>>>> Current release we haven't much in the cartridge heath monitoring but
>>>>> its very important and need to have in the future release.
>>>>>
>>>>> If we implemented above mentioned load average calculation mechanism
>>>> then we can do this easily.
>>>>
>>>
>>> Yes, health monitor would ideally consider the load average of the
>>> Cartridge instances, memory consumption etc. and transit the state
>>> according to a pre-defined curriculum, IMO.
>>>
>>> We can call that end point and get whatever we need. Also it would be
>>>> ideal if we can have some simple user interface for load balancer to view
>>>> active services and subscribed nodes and their status. Or else we can use
>>>> cluster message and set status of server as member attribute.
>>>>
>>>
>>>> Also i would like to suggest static load balance endpoint support for
>>>> load balancer(as a suggestion). Let say some user have fixed set of back
>>>> end servers and need to route traffic to them through load balancer. That
>>>> would be useful feature and we can consider it when we design milestone
>>>> plan.
>>>>
>>>
>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>> build, as and when required. We don't have a concept call static endpoints
>>> in the context of Stratos.
>>>
>> Yes i can understand. Thought it would be more useful if someone try to
>> load balance within static members.
>>
>>>
>>> Sample configuration would be as follows(It tells everything about
>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>> this.
>>>>
>>>> TestService {
>>>> hosts server.test.com;
>>>> servers {
>>>> server1 {
>>>> ip = 192.0.0.1;
>>>> http = 80;
>>>> https = 443;
>>>> }
>>>>
>>>> server2 {
>>>> ip = 192.0.0.2;
>>>> http = 80;
>>>> https = 443;
>>>> }
>>>> }
>>>> }
>>>>
>>>>>
>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>
>>>>> Current release we only support git based artifact distribution and
>>>>> need to provide more scalable deployment options
>>>>>
>>>>>
>>>>> - Light weight cartridge agent
>>>>>
>>>>> Current load balancer used tribes based clustering. To support tribes
>>>>> clustering we have to use java based cartridge agent and it will put some
>>>>> unnecessary heavy load to non java cartridges like PHP. If we can moved to
>>>>> Hazelcast Clustering we can have python based light weight cartridge agent
>>>>> across all type of cartridges.
>>>>>
>>>>>
>>>>> - Adding more and more cartridges
>>>>>
>>>>> We can have more and more cartridges like Python, Ruby, Node.js, Mongo
>>>>> etc.
>>>>>
>>>>>
>>>>> thanks
>>>>>
>>>>> [1]
>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>> [2]
>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>
>>>>>
>>>>> --
>>>>> Lakmal Warusawithana
>>>>> Software Architect; WSO2 Inc.
>>>>> Mobile : +94714289692
>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Sanjeewa Malalgoda
>>>> **B.Sc. Engineering(Hons)
>>>> Dip. in Com.Sc.
>>>> AMIESL , MIACSIT, CCNA
>>>>
>>>> *
>>>> Mobile +94713068779
>>>>
>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Nirmal
>>>
>>> C.S.Nirmal J. Fernando
>>> Senior Software Engineer,
>>> WSO2 Inc.
>>>
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>
>>
>> Thanks,
>> --
>>
>> *Sanjeewa Malalgoda**
>>
>> *
>> Mobile +94713068779
>>
>> http://sanjeewamalalgoda.blogspot.com/
>>
>
>
>
> --
> ...................................................
> Dharshana Kasun Warusavitharana
>
>
>
>
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: Stratos Milestone plan
Posted by Lahiru Sandaruwan <la...@wso2.com>.
Hi Sameera,
+1. I think this is our plan after we are done with re-factoring. We have
all the components in the base folder as per current plan. Coarse-grained
components is a must in this case.
Thanks.
On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <sameera.madushan@gmail.com
> wrote:
> Hi Lakmal
>
> IMHO we need to consider the option of having more coarse-grained
> components. If you look at the existing components and feature, there are
> too fine-grained. This is true even in the previous Stratos and Carbon code
> bases. Lets not make the same mistake again here. If we go ahead with
> fine-grained components then it will become a maintenance night-mare in the
> future.
>
> So I would suggest few logical components and few logical features(i.e
> Installable Units).
>
> Thanks,
> Sameera.
>
>
> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>
>> Hi all,
>>
>> I will start separate threads to discuss each item in detail. What I can
>> see milestone one should be, changing code to apache and re-factor the code
>> to new structure. After discuss in detail will prioritize and then finalize
>> the next milestone items.
>>
>> Current Stratos we use osgi model and will continue it. We will simplify
>> folder structure as below.
>>
>>
>> /
>> -Components
>> -- all osgi components
>> - Features (logically bundle components together)
>> -- adc
>> -- load balancer
>> -- auto scaller
>> -- manager
>> -- cloud controller
>> -- cli
>> -- cartridge agent
>> - Products (bundle several features and make a product)
>> - Service-stubs
>> -- all service stubs
>> - Tools
>> -- tools for create cartridges
>>
>>
>> Please share your thoughts.
>>
>>
>> thanks
>>
>>
>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>> dkasunw@gmail.com> wrote:
>>
>>> +1 for idea of load balance within static members. It allows to have
>>> more openings and be more flexible.
>>>
>>>
>>> Thank You,
>>> Dharshana.
>>>
>>>
>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <sanjeewa190@gmail.com
>>> > wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>> Hi Sanjeewa,
>>>>>
>>>>>
>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>> sanjeewa190@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>> lakmal@wso2.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I hope all commiters / folks are in the mailing list now. Here I
>>>>>>> have listed below, some identified improvements, features that like to have
>>>>>>> in Stratos.
>>>>>>>
>>>>>>> Any other improvements/features are mostly welcome and will add
>>>>>>> those to the bellow list, then we can discuss and then will create
>>>>>>> milestones plan for releases. ( We will commit current code based soon
>>>>>>> after we got the git repository available)
>>>>>>>
>>>>>>> Before that if some one want to get quick overview of current
>>>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>>>> more detail [2] with current product documentation, which will migrate to
>>>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>>>> architecture and code based, and it may helpful other commierts to come
>>>>>>> up-to speed quickly.
>>>>>>>
>>>>>>> Here the list;
>>>>>>>
>>>>>>> - Re-factor current code to Apache
>>>>>>>
>>>>>>> First we will move current code based to apache and then start
>>>>>>> re-factoring to apache and make a release without adding any other
>>>>>>> improvement or feature.
>>>>>>>
>>>>>>>
>>>>>>> - IaaS Independent cartridge for Linux.
>>>>>>>
>>>>>>> With the current architecture when you create a cartridge its depend
>>>>>>> on the IaaS. For example If you create a cartridge in EC2, we have to make
>>>>>>> an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>>>> start separate mail thread with more detail on this)
>>>>>>>
>>>>>>>
>>>>>>> - Support Windows based cartridges.
>>>>>>>
>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>
>>>>>>>
>>>>>>> - Expand auto-scaling algorithm
>>>>>>>
>>>>>>> Currently auto scaling decision take based on looking at length of
>>>>>>> the http/https request queue. We have to expand this to consider CPU usage,
>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>
>>>>>>> Before we introduce que length based decision making mechanism we
>>>>>> had load average based decision making mechanism. Each server need to
>>>>>> expose some API(some service) to get load average of given server. Load
>>>>>> calculation can be different from one server to other. Load balancer will
>>>>>> do web service call to those end points and update load average values of
>>>>>> member list periodically.
>>>>>>
>>>>>
>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>> logic soon. This would require some code level changes. But the vision is
>>>>> there.
>>>>>
>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>> components could provide/insert information that matters the decision of
>>>>> scaling up/down, depending on the algorithm used.
>>>>>
>>>>> Normally that service returns integer value(let say 0 to 10) and we
>>>>>> can get some idea about load from that. When it comes to traffic route we
>>>>>> will give high priority to server with minimum load and so on. So that type
>>>>>> of implementation would be ideal for this case as well. Synapse member
>>>>>> selection algorithm can be configurable and we can add new algorithm for
>>>>>> this if need.
>>>>>>
>>>>>
>>>>> This is an extension point, one could leverage.
>>>>>
>>>>>>
>>>>>>> - TCP level load balance.
>>>>>>>
>>>>>>> Current we only doing http/https based load balancing but need to
>>>>>>> expand this to do TCP level load balancing.
>>>>>>>
>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>> considerable amount of synapse transport level improvements to implement
>>>>>> this. WDYT?
>>>>>>
>>>>>
>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>> already available (I couldn't find any reference).
>>>>>
>>>>>>
>>>>>>> - Health monitoring
>>>>>>>
>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>> but its very important and need to have in the future release.
>>>>>>>
>>>>>>> If we implemented above mentioned load average calculation mechanism
>>>>>> then we can do this easily.
>>>>>>
>>>>>
>>>>> Yes, health monitor would ideally consider the load average of the
>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>> according to a pre-defined curriculum, IMO.
>>>>>
>>>>> We can call that end point and get whatever we need. Also it would be
>>>>>> ideal if we can have some simple user interface for load balancer to view
>>>>>> active services and subscribed nodes and their status. Or else we can use
>>>>>> cluster message and set status of server as member attribute.
>>>>>>
>>>>>
>>>>>> Also i would like to suggest static load balance endpoint support for
>>>>>> load balancer(as a suggestion). Let say some user have fixed set of back
>>>>>> end servers and need to route traffic to them through load balancer. That
>>>>>> would be useful feature and we can consider it when we design milestone
>>>>>> plan.
>>>>>>
>>>>>
>>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>>>> build, as and when required. We don't have a concept call static endpoints
>>>>> in the context of Stratos.
>>>>>
>>>> Yes i can understand. Thought it would be more useful if someone try to
>>>> load balance within static members.
>>>>
>>>>>
>>>>> Sample configuration would be as follows(It tells everything about
>>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>>> this.
>>>>>>
>>>>>> TestService {
>>>>>> hosts server.test.com;
>>>>>> servers {
>>>>>> server1 {
>>>>>> ip = 192.0.0.1;
>>>>>> http = 80;
>>>>>> https = 443;
>>>>>> }
>>>>>>
>>>>>> server2 {
>>>>>> ip = 192.0.0.2;
>>>>>> http = 80;
>>>>>> https = 443;
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>>
>>>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>
>>>>>>> Current release we only support git based artifact distribution and
>>>>>>> need to provide more scalable deployment options
>>>>>>>
>>>>>>>
>>>>>>> - Light weight cartridge agent
>>>>>>>
>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>> tribes clustering we have to use java based cartridge agent and it will put
>>>>>>> some unnecessary heavy load to non java cartridges like PHP. If we can
>>>>>>> moved to Hazelcast Clustering we can have python based light weight
>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>
>>>>>>>
>>>>>>> - Adding more and more cartridges
>>>>>>>
>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>> Mongo etc.
>>>>>>>
>>>>>>>
>>>>>>> thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>> [2]
>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Lakmal Warusawithana
>>>>>>> Software Architect; WSO2 Inc.
>>>>>>> Mobile : +94714289692
>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Sanjeewa Malalgoda
>>>>>> **B.Sc. Engineering(Hons)
>>>>>> Dip. in Com.Sc.
>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>
>>>>>> *
>>>>>> Mobile +94713068779
>>>>>>
>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> C.S.Nirmal J. Fernando
>>>>> Senior Software Engineer,
>>>>> WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>> Thanks,
>>>> --
>>>>
>>>> *Sanjeewa Malalgoda**
>>>>
>>>> *
>>>> Mobile +94713068779
>>>>
>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> ...................................................
>>> Dharshana Kasun Warusavitharana
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Sameera Jayasoma
>
> blog: http://sameera.adahas.org
> twitter: https://twitter.com/sameerajayasoma
> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>
--
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
Re: Stratos Milestone plan
Posted by Lakmal Warusawithana <la...@wso2.com>.
Yeah that will make life easier.
On Sun, Jul 7, 2013 at 6:00 PM, Isuru Haththotuwa <is...@wso2.com> wrote:
>
> On Sun, Jul 7, 2013 at 5:50 PM, Shariq Muhammed <sh...@wso2.com> wrote:
>
>> Would it also make sense to have a /dependencies project where we can
>> merge the libs coming from carbon kernel, there we can group stuff user-
>> mgt, registry, and kernel (carbon.core, carbon.utils etc). One advantage
>> is it would be much easier for developers to have one or two coarse-grained
>> libs instead of the many jars in the classpath, just a thought .. !
>>
>
> +1. We can make dependencies as well as the components more coarse grained
> to avoid unwanted complexity.
>
>>
>>
>> On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>>
>>> Hi Sameera,
>>>
>>> Yes have to create more coarse-grained components. Will do after the
>>> initial refactor completed.
>>>
>>> thanks
>>>
>>>
>>> On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <
>>> sameera.madushan@gmail.com> wrote:
>>>
>>>> Hi Lakmal
>>>>
>>>> IMHO we need to consider the option of having more coarse-grained
>>>> components. If you look at the existing components and feature, there are
>>>> too fine-grained. This is true even in the previous Stratos and Carbon code
>>>> bases. Lets not make the same mistake again here. If we go ahead with
>>>> fine-grained components then it will become a maintenance night-mare in the
>>>> future.
>>>>
>>>> So I would suggest few logical components and few logical features(i.e
>>>> Installable Units).
>>>>
>>>> Thanks,
>>>> Sameera.
>>>>
>>>>
>>>> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I will start separate threads to discuss each item in detail. What I
>>>>> can see milestone one should be, changing code to apache and re-factor the
>>>>> code to new structure. After discuss in detail will prioritize and then
>>>>> finalize the next milestone items.
>>>>>
>>>>> Current Stratos we use osgi model and will continue it. We will
>>>>> simplify folder structure as below.
>>>>>
>>>>>
>>>>> /
>>>>> -Components
>>>>> -- all osgi components
>>>>> - Features (logically bundle components together)
>>>>> -- adc
>>>>> -- load balancer
>>>>> -- auto scaller
>>>>> -- manager
>>>>> -- cloud controller
>>>>> -- cli
>>>>> -- cartridge agent
>>>>> - Products (bundle several features and make a product)
>>>>> - Service-stubs
>>>>> -- all service stubs
>>>>> - Tools
>>>>> -- tools for create cartridges
>>>>>
>>>>>
>>>>> Please share your thoughts.
>>>>>
>>>>>
>>>>> thanks
>>>>>
>>>>>
>>>>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>>>>> dkasunw@gmail.com> wrote:
>>>>>
>>>>>> +1 for idea of load balance within static members. It allows to have
>>>>>> more openings and be more flexible.
>>>>>>
>>>>>>
>>>>>> Thank You,
>>>>>> Dharshana.
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <
>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Sanjeewa,
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I hope all commiters / folks are in the mailing list now. Here I
>>>>>>>>>> have listed below, some identified improvements, features that like to have
>>>>>>>>>> in Stratos.
>>>>>>>>>>
>>>>>>>>>> Any other improvements/features are mostly welcome and will add
>>>>>>>>>> those to the bellow list, then we can discuss and then will create
>>>>>>>>>> milestones plan for releases. ( We will commit current code based soon
>>>>>>>>>> after we got the git repository available)
>>>>>>>>>>
>>>>>>>>>> Before that if some one want to get quick overview of current
>>>>>>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>>>>>>> more detail [2] with current product documentation, which will migrate to
>>>>>>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>>>>>>> architecture and code based, and it may helpful other commierts to come
>>>>>>>>>> up-to speed quickly.
>>>>>>>>>>
>>>>>>>>>> Here the list;
>>>>>>>>>>
>>>>>>>>>> - Re-factor current code to Apache
>>>>>>>>>>
>>>>>>>>>> First we will move current code based to apache and then start
>>>>>>>>>> re-factoring to apache and make a release without adding any other
>>>>>>>>>> improvement or feature.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - IaaS Independent cartridge for Linux.
>>>>>>>>>>
>>>>>>>>>> With the current architecture when you create a cartridge its
>>>>>>>>>> depend on the IaaS. For example If you create a cartridge in EC2, we have
>>>>>>>>>> to make an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>>>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>>>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>>>>>>> start separate mail thread with more detail on this)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Support Windows based cartridges.
>>>>>>>>>>
>>>>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Expand auto-scaling algorithm
>>>>>>>>>>
>>>>>>>>>> Currently auto scaling decision take based on looking at length
>>>>>>>>>> of the http/https request queue. We have to expand this to consider CPU
>>>>>>>>>> usage, memory utilization ..etc of the cartridges.
>>>>>>>>>>
>>>>>>>>>> Before we introduce que length based decision making mechanism we
>>>>>>>>> had load average based decision making mechanism. Each server need to
>>>>>>>>> expose some API(some service) to get load average of given server. Load
>>>>>>>>> calculation can be different from one server to other. Load balancer will
>>>>>>>>> do web service call to those end points and update load average values of
>>>>>>>>> member list periodically.
>>>>>>>>>
>>>>>>>>
>>>>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>>>>> logic soon. This would require some code level changes. But the vision is
>>>>>>>> there.
>>>>>>>>
>>>>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>>>>> components could provide/insert information that matters the decision of
>>>>>>>> scaling up/down, depending on the algorithm used.
>>>>>>>>
>>>>>>>> Normally that service returns integer value(let say 0 to 10) and
>>>>>>>>> we can get some idea about load from that. When it comes to traffic route
>>>>>>>>> we will give high priority to server with minimum load and so on. So that
>>>>>>>>> type of implementation would be ideal for this case as well. Synapse member
>>>>>>>>> selection algorithm can be configurable and we can add new algorithm for
>>>>>>>>> this if need.
>>>>>>>>>
>>>>>>>>
>>>>>>>> This is an extension point, one could leverage.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> - TCP level load balance.
>>>>>>>>>>
>>>>>>>>>> Current we only doing http/https based load balancing but need to
>>>>>>>>>> expand this to do TCP level load balancing.
>>>>>>>>>>
>>>>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>>>>> considerable amount of synapse transport level improvements to implement
>>>>>>>>> this. WDYT?
>>>>>>>>>
>>>>>>>>
>>>>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>>>>> already available (I couldn't find any reference).
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> - Health monitoring
>>>>>>>>>>
>>>>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>>>>> but its very important and need to have in the future release.
>>>>>>>>>>
>>>>>>>>>> If we implemented above mentioned load average calculation
>>>>>>>>> mechanism then we can do this easily.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, health monitor would ideally consider the load average of the
>>>>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>>>>> according to a pre-defined curriculum, IMO.
>>>>>>>>
>>>>>>>> We can call that end point and get whatever we need. Also it would
>>>>>>>>> be ideal if we can have some simple user interface for load balancer to
>>>>>>>>> view active services and subscribed nodes and their status. Or else we can
>>>>>>>>> use cluster message and set status of server as member attribute.
>>>>>>>>>
>>>>>>>>
>>>>>>>>> Also i would like to suggest static load balance endpoint support
>>>>>>>>> for load balancer(as a suggestion). Let say some user have fixed set of
>>>>>>>>> back end servers and need to route traffic to them through load balancer.
>>>>>>>>> That would be useful feature and we can consider it when we design
>>>>>>>>> milestone plan.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I'm afraid this won't be a use-case when you consider Apache
>>>>>>>> Stratos. Apache Stratos is a fully dynamic environment i.e. all the
>>>>>>>> clusters get build, as and when required. We don't have a concept call
>>>>>>>> static endpoints in the context of Stratos.
>>>>>>>>
>>>>>>> Yes i can understand. Thought it would be more useful if someone try
>>>>>>> to load balance within static members.
>>>>>>>
>>>>>>>>
>>>>>>>> Sample configuration would be as follows(It tells everything about
>>>>>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>>>>>> this.
>>>>>>>>>
>>>>>>>>> TestService {
>>>>>>>>> hosts server.test.com;
>>>>>>>>> servers {
>>>>>>>>> server1 {
>>>>>>>>> ip = 192.0.0.1;
>>>>>>>>> http = 80;
>>>>>>>>> https = 443;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> server2 {
>>>>>>>>> ip = 192.0.0.2;
>>>>>>>>> http = 80;
>>>>>>>>> https = 443;
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>>>>
>>>>>>>>>> Current release we only support git based artifact distribution
>>>>>>>>>> and need to provide more scalable deployment options
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Light weight cartridge agent
>>>>>>>>>>
>>>>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>>>>> tribes clustering we have to use java based cartridge agent and it will put
>>>>>>>>>> some unnecessary heavy load to non java cartridges like PHP. If we can
>>>>>>>>>> moved to Hazelcast Clustering we can have python based light weight
>>>>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Adding more and more cartridges
>>>>>>>>>>
>>>>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>>>>> Mongo etc.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> thanks
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>>>>> [2]
>>>>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Lakmal Warusawithana
>>>>>>>>>> Software Architect; WSO2 Inc.
>>>>>>>>>> Mobile : +94714289692
>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> *Sanjeewa Malalgoda
>>>>>>>>> **B.Sc. Engineering(Hons)
>>>>>>>>> Dip. in Com.Sc.
>>>>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>>>>
>>>>>>>>> *
>>>>>>>>> Mobile +94713068779
>>>>>>>>>
>>>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards,
>>>>>>>> Nirmal
>>>>>>>>
>>>>>>>> C.S.Nirmal J. Fernando
>>>>>>>> Senior Software Engineer,
>>>>>>>> WSO2 Inc.
>>>>>>>>
>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> --
>>>>>>>
>>>>>>> *Sanjeewa Malalgoda**
>>>>>>>
>>>>>>> *
>>>>>>> Mobile +94713068779
>>>>>>>
>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ...................................................
>>>>>> Dharshana Kasun Warusavitharana
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lakmal Warusawithana
>>>>> Software Architect; WSO2 Inc.
>>>>> Mobile : +94714289692
>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Sameera Jayasoma
>>>>
>>>> blog: http://sameera.adahas.org
>>>> twitter: https://twitter.com/sameerajayasoma
>>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>>>>
>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Software Architect; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Thanks,
>> Shariq.
>> Phone: +94 777 202 225
>>
>
>
>
> --
> Thanks and Regards,
>
> Isuru H.
>
>
>
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: Stratos Milestone plan
Posted by Isuru Haththotuwa <is...@wso2.com>.
On Sun, Jul 7, 2013 at 5:50 PM, Shariq Muhammed <sh...@wso2.com> wrote:
> Would it also make sense to have a /dependencies project where we can
> merge the libs coming from carbon kernel, there we can group stuff user-
> mgt, registry, and kernel (carbon.core, carbon.utils etc). One advantage
> is it would be much easier for developers to have one or two coarse-grained
> libs instead of the many jars in the classpath, just a thought .. !
>
+1. We can make dependencies as well as the components more coarse grained
to avoid unwanted complexity.
>
>
> On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>
>> Hi Sameera,
>>
>> Yes have to create more coarse-grained components. Will do after the
>> initial refactor completed.
>>
>> thanks
>>
>>
>> On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <
>> sameera.madushan@gmail.com> wrote:
>>
>>> Hi Lakmal
>>>
>>> IMHO we need to consider the option of having more coarse-grained
>>> components. If you look at the existing components and feature, there are
>>> too fine-grained. This is true even in the previous Stratos and Carbon code
>>> bases. Lets not make the same mistake again here. If we go ahead with
>>> fine-grained components then it will become a maintenance night-mare in the
>>> future.
>>>
>>> So I would suggest few logical components and few logical features(i.e
>>> Installable Units).
>>>
>>> Thanks,
>>> Sameera.
>>>
>>>
>>> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I will start separate threads to discuss each item in detail. What I
>>>> can see milestone one should be, changing code to apache and re-factor the
>>>> code to new structure. After discuss in detail will prioritize and then
>>>> finalize the next milestone items.
>>>>
>>>> Current Stratos we use osgi model and will continue it. We will
>>>> simplify folder structure as below.
>>>>
>>>>
>>>> /
>>>> -Components
>>>> -- all osgi components
>>>> - Features (logically bundle components together)
>>>> -- adc
>>>> -- load balancer
>>>> -- auto scaller
>>>> -- manager
>>>> -- cloud controller
>>>> -- cli
>>>> -- cartridge agent
>>>> - Products (bundle several features and make a product)
>>>> - Service-stubs
>>>> -- all service stubs
>>>> - Tools
>>>> -- tools for create cartridges
>>>>
>>>>
>>>> Please share your thoughts.
>>>>
>>>>
>>>> thanks
>>>>
>>>>
>>>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>>>> dkasunw@gmail.com> wrote:
>>>>
>>>>> +1 for idea of load balance within static members. It allows to have
>>>>> more openings and be more flexible.
>>>>>
>>>>>
>>>>> Thank You,
>>>>> Dharshana.
>>>>>
>>>>>
>>>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <
>>>>> sanjeewa190@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Sanjeewa,
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I hope all commiters / folks are in the mailing list now. Here I
>>>>>>>>> have listed below, some identified improvements, features that like to have
>>>>>>>>> in Stratos.
>>>>>>>>>
>>>>>>>>> Any other improvements/features are mostly welcome and will add
>>>>>>>>> those to the bellow list, then we can discuss and then will create
>>>>>>>>> milestones plan for releases. ( We will commit current code based soon
>>>>>>>>> after we got the git repository available)
>>>>>>>>>
>>>>>>>>> Before that if some one want to get quick overview of current
>>>>>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>>>>>> more detail [2] with current product documentation, which will migrate to
>>>>>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>>>>>> architecture and code based, and it may helpful other commierts to come
>>>>>>>>> up-to speed quickly.
>>>>>>>>>
>>>>>>>>> Here the list;
>>>>>>>>>
>>>>>>>>> - Re-factor current code to Apache
>>>>>>>>>
>>>>>>>>> First we will move current code based to apache and then start
>>>>>>>>> re-factoring to apache and make a release without adding any other
>>>>>>>>> improvement or feature.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - IaaS Independent cartridge for Linux.
>>>>>>>>>
>>>>>>>>> With the current architecture when you create a cartridge its
>>>>>>>>> depend on the IaaS. For example If you create a cartridge in EC2, we have
>>>>>>>>> to make an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>>>>>> start separate mail thread with more detail on this)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Support Windows based cartridges.
>>>>>>>>>
>>>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Expand auto-scaling algorithm
>>>>>>>>>
>>>>>>>>> Currently auto scaling decision take based on looking at length of
>>>>>>>>> the http/https request queue. We have to expand this to consider CPU usage,
>>>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>>>
>>>>>>>>> Before we introduce que length based decision making mechanism we
>>>>>>>> had load average based decision making mechanism. Each server need to
>>>>>>>> expose some API(some service) to get load average of given server. Load
>>>>>>>> calculation can be different from one server to other. Load balancer will
>>>>>>>> do web service call to those end points and update load average values of
>>>>>>>> member list periodically.
>>>>>>>>
>>>>>>>
>>>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>>>> logic soon. This would require some code level changes. But the vision is
>>>>>>> there.
>>>>>>>
>>>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>>>> components could provide/insert information that matters the decision of
>>>>>>> scaling up/down, depending on the algorithm used.
>>>>>>>
>>>>>>> Normally that service returns integer value(let say 0 to 10) and
>>>>>>>> we can get some idea about load from that. When it comes to traffic route
>>>>>>>> we will give high priority to server with minimum load and so on. So that
>>>>>>>> type of implementation would be ideal for this case as well. Synapse member
>>>>>>>> selection algorithm can be configurable and we can add new algorithm for
>>>>>>>> this if need.
>>>>>>>>
>>>>>>>
>>>>>>> This is an extension point, one could leverage.
>>>>>>>
>>>>>>>>
>>>>>>>>> - TCP level load balance.
>>>>>>>>>
>>>>>>>>> Current we only doing http/https based load balancing but need to
>>>>>>>>> expand this to do TCP level load balancing.
>>>>>>>>>
>>>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>>>> considerable amount of synapse transport level improvements to implement
>>>>>>>> this. WDYT?
>>>>>>>>
>>>>>>>
>>>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>>>> already available (I couldn't find any reference).
>>>>>>>
>>>>>>>>
>>>>>>>>> - Health monitoring
>>>>>>>>>
>>>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>>>> but its very important and need to have in the future release.
>>>>>>>>>
>>>>>>>>> If we implemented above mentioned load average calculation
>>>>>>>> mechanism then we can do this easily.
>>>>>>>>
>>>>>>>
>>>>>>> Yes, health monitor would ideally consider the load average of the
>>>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>>>> according to a pre-defined curriculum, IMO.
>>>>>>>
>>>>>>> We can call that end point and get whatever we need. Also it would
>>>>>>>> be ideal if we can have some simple user interface for load balancer to
>>>>>>>> view active services and subscribed nodes and their status. Or else we can
>>>>>>>> use cluster message and set status of server as member attribute.
>>>>>>>>
>>>>>>>
>>>>>>>> Also i would like to suggest static load balance endpoint support
>>>>>>>> for load balancer(as a suggestion). Let say some user have fixed set of
>>>>>>>> back end servers and need to route traffic to them through load balancer.
>>>>>>>> That would be useful feature and we can consider it when we design
>>>>>>>> milestone plan.
>>>>>>>>
>>>>>>>
>>>>>>> I'm afraid this won't be a use-case when you consider Apache
>>>>>>> Stratos. Apache Stratos is a fully dynamic environment i.e. all the
>>>>>>> clusters get build, as and when required. We don't have a concept call
>>>>>>> static endpoints in the context of Stratos.
>>>>>>>
>>>>>> Yes i can understand. Thought it would be more useful if someone try
>>>>>> to load balance within static members.
>>>>>>
>>>>>>>
>>>>>>> Sample configuration would be as follows(It tells everything about
>>>>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>>>>> this.
>>>>>>>>
>>>>>>>> TestService {
>>>>>>>> hosts server.test.com;
>>>>>>>> servers {
>>>>>>>> server1 {
>>>>>>>> ip = 192.0.0.1;
>>>>>>>> http = 80;
>>>>>>>> https = 443;
>>>>>>>> }
>>>>>>>>
>>>>>>>> server2 {
>>>>>>>> ip = 192.0.0.2;
>>>>>>>> http = 80;
>>>>>>>> https = 443;
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>>>
>>>>>>>>> Current release we only support git based artifact distribution
>>>>>>>>> and need to provide more scalable deployment options
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Light weight cartridge agent
>>>>>>>>>
>>>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>>>> tribes clustering we have to use java based cartridge agent and it will put
>>>>>>>>> some unnecessary heavy load to non java cartridges like PHP. If we can
>>>>>>>>> moved to Hazelcast Clustering we can have python based light weight
>>>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Adding more and more cartridges
>>>>>>>>>
>>>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>>>> Mongo etc.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> thanks
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>>>> [2]
>>>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Lakmal Warusawithana
>>>>>>>>> Software Architect; WSO2 Inc.
>>>>>>>>> Mobile : +94714289692
>>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> *Sanjeewa Malalgoda
>>>>>>>> **B.Sc. Engineering(Hons)
>>>>>>>> Dip. in Com.Sc.
>>>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>>>
>>>>>>>> *
>>>>>>>> Mobile +94713068779
>>>>>>>>
>>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Nirmal
>>>>>>>
>>>>>>> C.S.Nirmal J. Fernando
>>>>>>> Senior Software Engineer,
>>>>>>> WSO2 Inc.
>>>>>>>
>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> --
>>>>>>
>>>>>> *Sanjeewa Malalgoda**
>>>>>>
>>>>>> *
>>>>>> Mobile +94713068779
>>>>>>
>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ...................................................
>>>>> Dharshana Kasun Warusavitharana
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Lakmal Warusawithana
>>>> Software Architect; WSO2 Inc.
>>>> Mobile : +94714289692
>>>> Blog : http://lakmalsview.blogspot.com/
>>>>
>>>>
>>>
>>>
>>> --
>>> Sameera Jayasoma
>>>
>>> blog: http://sameera.adahas.org
>>> twitter: https://twitter.com/sameerajayasoma
>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Thanks,
> Shariq.
> Phone: +94 777 202 225
>
--
Thanks and Regards,
Isuru H.
Re: Stratos Milestone plan
Posted by Shariq Muhammed <sh...@wso2.com>.
Would it also make sense to have a /dependencies project where we can merge
the libs coming from carbon kernel, there we can group stuff user-mgt,
registry, and kernel (carbon.core, carbon.utils etc). One advantage is it
would be much easier for developers to have one or two coarse-grained libs
instead of the many jars in the classpath, just a thought .. !
On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
> Hi Sameera,
>
> Yes have to create more coarse-grained components. Will do after the
> initial refactor completed.
>
> thanks
>
>
> On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <
> sameera.madushan@gmail.com> wrote:
>
>> Hi Lakmal
>>
>> IMHO we need to consider the option of having more coarse-grained
>> components. If you look at the existing components and feature, there are
>> too fine-grained. This is true even in the previous Stratos and Carbon code
>> bases. Lets not make the same mistake again here. If we go ahead with
>> fine-grained components then it will become a maintenance night-mare in the
>> future.
>>
>> So I would suggest few logical components and few logical features(i.e
>> Installable Units).
>>
>> Thanks,
>> Sameera.
>>
>>
>> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>>
>>> Hi all,
>>>
>>> I will start separate threads to discuss each item in detail. What I can
>>> see milestone one should be, changing code to apache and re-factor the code
>>> to new structure. After discuss in detail will prioritize and then finalize
>>> the next milestone items.
>>>
>>> Current Stratos we use osgi model and will continue it. We will simplify
>>> folder structure as below.
>>>
>>>
>>> /
>>> -Components
>>> -- all osgi components
>>> - Features (logically bundle components together)
>>> -- adc
>>> -- load balancer
>>> -- auto scaller
>>> -- manager
>>> -- cloud controller
>>> -- cli
>>> -- cartridge agent
>>> - Products (bundle several features and make a product)
>>> - Service-stubs
>>> -- all service stubs
>>> - Tools
>>> -- tools for create cartridges
>>>
>>>
>>> Please share your thoughts.
>>>
>>>
>>> thanks
>>>
>>>
>>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>>> dkasunw@gmail.com> wrote:
>>>
>>>> +1 for idea of load balance within static members. It allows to have
>>>> more openings and be more flexible.
>>>>
>>>>
>>>> Thank You,
>>>> Dharshana.
>>>>
>>>>
>>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <
>>>> sanjeewa190@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>>> nirmal070125@gmail.com> wrote:
>>>>>
>>>>>> Hi Sanjeewa,
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I hope all commiters / folks are in the mailing list now. Here I
>>>>>>>> have listed below, some identified improvements, features that like to have
>>>>>>>> in Stratos.
>>>>>>>>
>>>>>>>> Any other improvements/features are mostly welcome and will add
>>>>>>>> those to the bellow list, then we can discuss and then will create
>>>>>>>> milestones plan for releases. ( We will commit current code based soon
>>>>>>>> after we got the git repository available)
>>>>>>>>
>>>>>>>> Before that if some one want to get quick overview of current
>>>>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>>>>> more detail [2] with current product documentation, which will migrate to
>>>>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>>>>> architecture and code based, and it may helpful other commierts to come
>>>>>>>> up-to speed quickly.
>>>>>>>>
>>>>>>>> Here the list;
>>>>>>>>
>>>>>>>> - Re-factor current code to Apache
>>>>>>>>
>>>>>>>> First we will move current code based to apache and then start
>>>>>>>> re-factoring to apache and make a release without adding any other
>>>>>>>> improvement or feature.
>>>>>>>>
>>>>>>>>
>>>>>>>> - IaaS Independent cartridge for Linux.
>>>>>>>>
>>>>>>>> With the current architecture when you create a cartridge its
>>>>>>>> depend on the IaaS. For example If you create a cartridge in EC2, we have
>>>>>>>> to make an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>>>>> start separate mail thread with more detail on this)
>>>>>>>>
>>>>>>>>
>>>>>>>> - Support Windows based cartridges.
>>>>>>>>
>>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>>
>>>>>>>>
>>>>>>>> - Expand auto-scaling algorithm
>>>>>>>>
>>>>>>>> Currently auto scaling decision take based on looking at length of
>>>>>>>> the http/https request queue. We have to expand this to consider CPU usage,
>>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>>
>>>>>>>> Before we introduce que length based decision making mechanism we
>>>>>>> had load average based decision making mechanism. Each server need to
>>>>>>> expose some API(some service) to get load average of given server. Load
>>>>>>> calculation can be different from one server to other. Load balancer will
>>>>>>> do web service call to those end points and update load average values of
>>>>>>> member list periodically.
>>>>>>>
>>>>>>
>>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>>> logic soon. This would require some code level changes. But the vision is
>>>>>> there.
>>>>>>
>>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>>> components could provide/insert information that matters the decision of
>>>>>> scaling up/down, depending on the algorithm used.
>>>>>>
>>>>>> Normally that service returns integer value(let say 0 to 10) and we
>>>>>>> can get some idea about load from that. When it comes to traffic route we
>>>>>>> will give high priority to server with minimum load and so on. So that type
>>>>>>> of implementation would be ideal for this case as well. Synapse member
>>>>>>> selection algorithm can be configurable and we can add new algorithm for
>>>>>>> this if need.
>>>>>>>
>>>>>>
>>>>>> This is an extension point, one could leverage.
>>>>>>
>>>>>>>
>>>>>>>> - TCP level load balance.
>>>>>>>>
>>>>>>>> Current we only doing http/https based load balancing but need to
>>>>>>>> expand this to do TCP level load balancing.
>>>>>>>>
>>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>>> considerable amount of synapse transport level improvements to implement
>>>>>>> this. WDYT?
>>>>>>>
>>>>>>
>>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>>> already available (I couldn't find any reference).
>>>>>>
>>>>>>>
>>>>>>>> - Health monitoring
>>>>>>>>
>>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>>> but its very important and need to have in the future release.
>>>>>>>>
>>>>>>>> If we implemented above mentioned load average calculation
>>>>>>> mechanism then we can do this easily.
>>>>>>>
>>>>>>
>>>>>> Yes, health monitor would ideally consider the load average of the
>>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>>> according to a pre-defined curriculum, IMO.
>>>>>>
>>>>>> We can call that end point and get whatever we need. Also it would
>>>>>>> be ideal if we can have some simple user interface for load balancer to
>>>>>>> view active services and subscribed nodes and their status. Or else we can
>>>>>>> use cluster message and set status of server as member attribute.
>>>>>>>
>>>>>>
>>>>>>> Also i would like to suggest static load balance endpoint support
>>>>>>> for load balancer(as a suggestion). Let say some user have fixed set of
>>>>>>> back end servers and need to route traffic to them through load balancer.
>>>>>>> That would be useful feature and we can consider it when we design
>>>>>>> milestone plan.
>>>>>>>
>>>>>>
>>>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>>>>> build, as and when required. We don't have a concept call static endpoints
>>>>>> in the context of Stratos.
>>>>>>
>>>>> Yes i can understand. Thought it would be more useful if someone try
>>>>> to load balance within static members.
>>>>>
>>>>>>
>>>>>> Sample configuration would be as follows(It tells everything about
>>>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>>>> this.
>>>>>>>
>>>>>>> TestService {
>>>>>>> hosts server.test.com;
>>>>>>> servers {
>>>>>>> server1 {
>>>>>>> ip = 192.0.0.1;
>>>>>>> http = 80;
>>>>>>> https = 443;
>>>>>>> }
>>>>>>>
>>>>>>> server2 {
>>>>>>> ip = 192.0.0.2;
>>>>>>> http = 80;
>>>>>>> https = 443;
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>>
>>>>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>>
>>>>>>>> Current release we only support git based artifact distribution and
>>>>>>>> need to provide more scalable deployment options
>>>>>>>>
>>>>>>>>
>>>>>>>> - Light weight cartridge agent
>>>>>>>>
>>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>>> tribes clustering we have to use java based cartridge agent and it will put
>>>>>>>> some unnecessary heavy load to non java cartridges like PHP. If we can
>>>>>>>> moved to Hazelcast Clustering we can have python based light weight
>>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>>
>>>>>>>>
>>>>>>>> - Adding more and more cartridges
>>>>>>>>
>>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>>> Mongo etc.
>>>>>>>>
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>>> [2]
>>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Lakmal Warusawithana
>>>>>>>> Software Architect; WSO2 Inc.
>>>>>>>> Mobile : +94714289692
>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Sanjeewa Malalgoda
>>>>>>> **B.Sc. Engineering(Hons)
>>>>>>> Dip. in Com.Sc.
>>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>>
>>>>>>> *
>>>>>>> Mobile +94713068779
>>>>>>>
>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Nirmal
>>>>>>
>>>>>> C.S.Nirmal J. Fernando
>>>>>> Senior Software Engineer,
>>>>>> WSO2 Inc.
>>>>>>
>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>> --
>>>>>
>>>>> *Sanjeewa Malalgoda**
>>>>>
>>>>> *
>>>>> Mobile +94713068779
>>>>>
>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ...................................................
>>>> Dharshana Kasun Warusavitharana
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Software Architect; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Sameera Jayasoma
>>
>> blog: http://sameera.adahas.org
>> twitter: https://twitter.com/sameerajayasoma
>> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>>
>
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
--
Thanks,
Shariq.
Phone: +94 777 202 225
Re: Stratos Milestone plan
Posted by Lakmal Warusawithana <la...@wso2.com>.
Hi Sameera,
Yes have to create more coarse-grained components. Will do after the
initial refactor completed.
thanks
On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <sameera.madushan@gmail.com
> wrote:
> Hi Lakmal
>
> IMHO we need to consider the option of having more coarse-grained
> components. If you look at the existing components and feature, there are
> too fine-grained. This is true even in the previous Stratos and Carbon code
> bases. Lets not make the same mistake again here. If we go ahead with
> fine-grained components then it will become a maintenance night-mare in the
> future.
>
> So I would suggest few logical components and few logical features(i.e
> Installable Units).
>
> Thanks,
> Sameera.
>
>
> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
>
>> Hi all,
>>
>> I will start separate threads to discuss each item in detail. What I can
>> see milestone one should be, changing code to apache and re-factor the code
>> to new structure. After discuss in detail will prioritize and then finalize
>> the next milestone items.
>>
>> Current Stratos we use osgi model and will continue it. We will simplify
>> folder structure as below.
>>
>>
>> /
>> -Components
>> -- all osgi components
>> - Features (logically bundle components together)
>> -- adc
>> -- load balancer
>> -- auto scaller
>> -- manager
>> -- cloud controller
>> -- cli
>> -- cartridge agent
>> - Products (bundle several features and make a product)
>> - Service-stubs
>> -- all service stubs
>> - Tools
>> -- tools for create cartridges
>>
>>
>> Please share your thoughts.
>>
>>
>> thanks
>>
>>
>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>> dkasunw@gmail.com> wrote:
>>
>>> +1 for idea of load balance within static members. It allows to have
>>> more openings and be more flexible.
>>>
>>>
>>> Thank You,
>>> Dharshana.
>>>
>>>
>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <sanjeewa190@gmail.com
>>> > wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>> Hi Sanjeewa,
>>>>>
>>>>>
>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>> sanjeewa190@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>> lakmal@wso2.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I hope all commiters / folks are in the mailing list now. Here I
>>>>>>> have listed below, some identified improvements, features that like to have
>>>>>>> in Stratos.
>>>>>>>
>>>>>>> Any other improvements/features are mostly welcome and will add
>>>>>>> those to the bellow list, then we can discuss and then will create
>>>>>>> milestones plan for releases. ( We will commit current code based soon
>>>>>>> after we got the git repository available)
>>>>>>>
>>>>>>> Before that if some one want to get quick overview of current
>>>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>>>> more detail [2] with current product documentation, which will migrate to
>>>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>>>> architecture and code based, and it may helpful other commierts to come
>>>>>>> up-to speed quickly.
>>>>>>>
>>>>>>> Here the list;
>>>>>>>
>>>>>>> - Re-factor current code to Apache
>>>>>>>
>>>>>>> First we will move current code based to apache and then start
>>>>>>> re-factoring to apache and make a release without adding any other
>>>>>>> improvement or feature.
>>>>>>>
>>>>>>>
>>>>>>> - IaaS Independent cartridge for Linux.
>>>>>>>
>>>>>>> With the current architecture when you create a cartridge its depend
>>>>>>> on the IaaS. For example If you create a cartridge in EC2, we have to make
>>>>>>> an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>>>> start separate mail thread with more detail on this)
>>>>>>>
>>>>>>>
>>>>>>> - Support Windows based cartridges.
>>>>>>>
>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>
>>>>>>>
>>>>>>> - Expand auto-scaling algorithm
>>>>>>>
>>>>>>> Currently auto scaling decision take based on looking at length of
>>>>>>> the http/https request queue. We have to expand this to consider CPU usage,
>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>
>>>>>>> Before we introduce que length based decision making mechanism we
>>>>>> had load average based decision making mechanism. Each server need to
>>>>>> expose some API(some service) to get load average of given server. Load
>>>>>> calculation can be different from one server to other. Load balancer will
>>>>>> do web service call to those end points and update load average values of
>>>>>> member list periodically.
>>>>>>
>>>>>
>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>> logic soon. This would require some code level changes. But the vision is
>>>>> there.
>>>>>
>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>> components could provide/insert information that matters the decision of
>>>>> scaling up/down, depending on the algorithm used.
>>>>>
>>>>> Normally that service returns integer value(let say 0 to 10) and we
>>>>>> can get some idea about load from that. When it comes to traffic route we
>>>>>> will give high priority to server with minimum load and so on. So that type
>>>>>> of implementation would be ideal for this case as well. Synapse member
>>>>>> selection algorithm can be configurable and we can add new algorithm for
>>>>>> this if need.
>>>>>>
>>>>>
>>>>> This is an extension point, one could leverage.
>>>>>
>>>>>>
>>>>>>> - TCP level load balance.
>>>>>>>
>>>>>>> Current we only doing http/https based load balancing but need to
>>>>>>> expand this to do TCP level load balancing.
>>>>>>>
>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>> considerable amount of synapse transport level improvements to implement
>>>>>> this. WDYT?
>>>>>>
>>>>>
>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>> already available (I couldn't find any reference).
>>>>>
>>>>>>
>>>>>>> - Health monitoring
>>>>>>>
>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>> but its very important and need to have in the future release.
>>>>>>>
>>>>>>> If we implemented above mentioned load average calculation mechanism
>>>>>> then we can do this easily.
>>>>>>
>>>>>
>>>>> Yes, health monitor would ideally consider the load average of the
>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>> according to a pre-defined curriculum, IMO.
>>>>>
>>>>> We can call that end point and get whatever we need. Also it would be
>>>>>> ideal if we can have some simple user interface for load balancer to view
>>>>>> active services and subscribed nodes and their status. Or else we can use
>>>>>> cluster message and set status of server as member attribute.
>>>>>>
>>>>>
>>>>>> Also i would like to suggest static load balance endpoint support for
>>>>>> load balancer(as a suggestion). Let say some user have fixed set of back
>>>>>> end servers and need to route traffic to them through load balancer. That
>>>>>> would be useful feature and we can consider it when we design milestone
>>>>>> plan.
>>>>>>
>>>>>
>>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>>>> build, as and when required. We don't have a concept call static endpoints
>>>>> in the context of Stratos.
>>>>>
>>>> Yes i can understand. Thought it would be more useful if someone try to
>>>> load balance within static members.
>>>>
>>>>>
>>>>> Sample configuration would be as follows(It tells everything about
>>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>>> this.
>>>>>>
>>>>>> TestService {
>>>>>> hosts server.test.com;
>>>>>> servers {
>>>>>> server1 {
>>>>>> ip = 192.0.0.1;
>>>>>> http = 80;
>>>>>> https = 443;
>>>>>> }
>>>>>>
>>>>>> server2 {
>>>>>> ip = 192.0.0.2;
>>>>>> http = 80;
>>>>>> https = 443;
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>>
>>>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>
>>>>>>> Current release we only support git based artifact distribution and
>>>>>>> need to provide more scalable deployment options
>>>>>>>
>>>>>>>
>>>>>>> - Light weight cartridge agent
>>>>>>>
>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>> tribes clustering we have to use java based cartridge agent and it will put
>>>>>>> some unnecessary heavy load to non java cartridges like PHP. If we can
>>>>>>> moved to Hazelcast Clustering we can have python based light weight
>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>
>>>>>>>
>>>>>>> - Adding more and more cartridges
>>>>>>>
>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>> Mongo etc.
>>>>>>>
>>>>>>>
>>>>>>> thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>> [2]
>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Lakmal Warusawithana
>>>>>>> Software Architect; WSO2 Inc.
>>>>>>> Mobile : +94714289692
>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Sanjeewa Malalgoda
>>>>>> **B.Sc. Engineering(Hons)
>>>>>> Dip. in Com.Sc.
>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>
>>>>>> *
>>>>>> Mobile +94713068779
>>>>>>
>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> C.S.Nirmal J. Fernando
>>>>> Senior Software Engineer,
>>>>> WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>> Thanks,
>>>> --
>>>>
>>>> *Sanjeewa Malalgoda**
>>>>
>>>> *
>>>> Mobile +94713068779
>>>>
>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> ...................................................
>>> Dharshana Kasun Warusavitharana
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Sameera Jayasoma
>
> blog: http://sameera.adahas.org
> twitter: https://twitter.com/sameerajayasoma
> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>
--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: Stratos Milestone plan
Posted by Sameera Jayasoma <sa...@gmail.com>.
Hi Lakmal
IMHO we need to consider the option of having more coarse-grained
components. If you look at the existing components and feature, there are
too fine-grained. This is true even in the previous Stratos and Carbon code
bases. Lets not make the same mistake again here. If we go ahead with
fine-grained components then it will become a maintenance night-mare in the
future.
So I would suggest few logical components and few logical features(i.e
Installable Units).
Thanks,
Sameera.
On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <la...@wso2.com>wrote:
> Hi all,
>
> I will start separate threads to discuss each item in detail. What I can
> see milestone one should be, changing code to apache and re-factor the code
> to new structure. After discuss in detail will prioritize and then finalize
> the next milestone items.
>
> Current Stratos we use osgi model and will continue it. We will simplify
> folder structure as below.
>
>
> /
> -Components
> -- all osgi components
> - Features (logically bundle components together)
> -- adc
> -- load balancer
> -- auto scaller
> -- manager
> -- cloud controller
> -- cli
> -- cartridge agent
> - Products (bundle several features and make a product)
> - Service-stubs
> -- all service stubs
> - Tools
> -- tools for create cartridges
>
>
> Please share your thoughts.
>
>
> thanks
>
>
> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
> dkasunw@gmail.com> wrote:
>
>> +1 for idea of load balance within static members. It allows to have
>> more openings and be more flexible.
>>
>>
>> Thank You,
>> Dharshana.
>>
>>
>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <sa...@gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>> nirmal070125@gmail.com> wrote:
>>>
>>>> Hi Sanjeewa,
>>>>
>>>>
>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>> sanjeewa190@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <lakmal@wso2.com
>>>>> > wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I hope all commiters / folks are in the mailing list now. Here I have
>>>>>> listed below, some identified improvements, features that like to have in
>>>>>> Stratos.
>>>>>>
>>>>>> Any other improvements/features are mostly welcome and will add those
>>>>>> to the bellow list, then we can discuss and then will create milestones
>>>>>> plan for releases. ( We will commit current code based soon after we got
>>>>>> the git repository available)
>>>>>>
>>>>>> Before that if some one want to get quick overview of current
>>>>>> release/architecture of Stratos please see [1] for my reason blog post or
>>>>>> more detail [2] with current product documentation, which will migrate to
>>>>>> Apache soon. Also we will organize some hangout sessions to explain current
>>>>>> architecture and code based, and it may helpful other commierts to come
>>>>>> up-to speed quickly.
>>>>>>
>>>>>> Here the list;
>>>>>>
>>>>>> - Re-factor current code to Apache
>>>>>>
>>>>>> First we will move current code based to apache and then start
>>>>>> re-factoring to apache and make a release without adding any other
>>>>>> improvement or feature.
>>>>>>
>>>>>>
>>>>>> - IaaS Independent cartridge for Linux.
>>>>>>
>>>>>> With the current architecture when you create a cartridge its depend
>>>>>> on the IaaS. For example If you create a cartridge in EC2, we have to make
>>>>>> an AMI cartridge. Likewise you have to create cartridges (say PHP
>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution we
>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) and
>>>>>> can be consider LXC is the default Stratos runtime container for Linux
>>>>>> cartridges. Also this will provide more scallability to the PaaS (I will
>>>>>> start separate mail thread with more detail on this)
>>>>>>
>>>>>>
>>>>>> - Support Windows based cartridges.
>>>>>>
>>>>>> If IaaS support windows VM then we can have windows based cartridges.
>>>>>> It will lead to have .net cartridges
>>>>>>
>>>>>>
>>>>>> - Expand auto-scaling algorithm
>>>>>>
>>>>>> Currently auto scaling decision take based on looking at length of
>>>>>> the http/https request queue. We have to expand this to consider CPU usage,
>>>>>> memory utilization ..etc of the cartridges.
>>>>>>
>>>>>> Before we introduce que length based decision making mechanism we had
>>>>> load average based decision making mechanism. Each server need to expose
>>>>> some API(some service) to get load average of given server. Load
>>>>> calculation can be different from one server to other. Load balancer will
>>>>> do web service call to those end points and update load average values of
>>>>> member list periodically.
>>>>>
>>>>
>>>> We will separate out the load balancer logic and the auto-scaling logic
>>>> soon. This would require some code level changes. But the vision is there.
>>>>
>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>> components could provide/insert information that matters the decision of
>>>> scaling up/down, depending on the algorithm used.
>>>>
>>>> Normally that service returns integer value(let say 0 to 10) and we
>>>>> can get some idea about load from that. When it comes to traffic route we
>>>>> will give high priority to server with minimum load and so on. So that type
>>>>> of implementation would be ideal for this case as well. Synapse member
>>>>> selection algorithm can be configurable and we can add new algorithm for
>>>>> this if need.
>>>>>
>>>>
>>>> This is an extension point, one could leverage.
>>>>
>>>>>
>>>>>> - TCP level load balance.
>>>>>>
>>>>>> Current we only doing http/https based load balancing but need to
>>>>>> expand this to do TCP level load balancing.
>>>>>>
>>>>>> This would be again nice feature IMO. Also i think we need
>>>>> considerable amount of synapse transport level improvements to implement
>>>>> this. WDYT?
>>>>>
>>>>
>>>> We may require to write a TCP load balance endpoint, if one is not
>>>> already available (I couldn't find any reference).
>>>>
>>>>>
>>>>>> - Health monitoring
>>>>>>
>>>>>> Current release we haven't much in the cartridge heath monitoring but
>>>>>> its very important and need to have in the future release.
>>>>>>
>>>>>> If we implemented above mentioned load average calculation mechanism
>>>>> then we can do this easily.
>>>>>
>>>>
>>>> Yes, health monitor would ideally consider the load average of the
>>>> Cartridge instances, memory consumption etc. and transit the state
>>>> according to a pre-defined curriculum, IMO.
>>>>
>>>> We can call that end point and get whatever we need. Also it would be
>>>>> ideal if we can have some simple user interface for load balancer to view
>>>>> active services and subscribed nodes and their status. Or else we can use
>>>>> cluster message and set status of server as member attribute.
>>>>>
>>>>
>>>>> Also i would like to suggest static load balance endpoint support for
>>>>> load balancer(as a suggestion). Let say some user have fixed set of back
>>>>> end servers and need to route traffic to them through load balancer. That
>>>>> would be useful feature and we can consider it when we design milestone
>>>>> plan.
>>>>>
>>>>
>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>>> build, as and when required. We don't have a concept call static endpoints
>>>> in the context of Stratos.
>>>>
>>> Yes i can understand. Thought it would be more useful if someone try to
>>> load balance within static members.
>>>
>>>>
>>>> Sample configuration would be as follows(It tells everything about
>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>> this.
>>>>>
>>>>> TestService {
>>>>> hosts server.test.com;
>>>>> servers {
>>>>> server1 {
>>>>> ip = 192.0.0.1;
>>>>> http = 80;
>>>>> https = 443;
>>>>> }
>>>>>
>>>>> server2 {
>>>>> ip = 192.0.0.2;
>>>>> http = 80;
>>>>> https = 443;
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>>>
>>>>>> - Improving ADC (Artifact Distribution Coordinator)
>>>>>>
>>>>>> Current release we only support git based artifact distribution and
>>>>>> need to provide more scalable deployment options
>>>>>>
>>>>>>
>>>>>> - Light weight cartridge agent
>>>>>>
>>>>>> Current load balancer used tribes based clustering. To support tribes
>>>>>> clustering we have to use java based cartridge agent and it will put some
>>>>>> unnecessary heavy load to non java cartridges like PHP. If we can moved to
>>>>>> Hazelcast Clustering we can have python based light weight cartridge agent
>>>>>> across all type of cartridges.
>>>>>>
>>>>>>
>>>>>> - Adding more and more cartridges
>>>>>>
>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>> Mongo etc.
>>>>>>
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>> [1]
>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>> [2]
>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Lakmal Warusawithana
>>>>>> Software Architect; WSO2 Inc.
>>>>>> Mobile : +94714289692
>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Sanjeewa Malalgoda
>>>>> **B.Sc. Engineering(Hons)
>>>>> Dip. in Com.Sc.
>>>>> AMIESL , MIACSIT, CCNA
>>>>>
>>>>> *
>>>>> Mobile +94713068779
>>>>>
>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Nirmal
>>>>
>>>> C.S.Nirmal J. Fernando
>>>> Senior Software Engineer,
>>>> WSO2 Inc.
>>>>
>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>
>>>
>>>
>>> Thanks,
>>> --
>>>
>>> *Sanjeewa Malalgoda**
>>>
>>> *
>>> Mobile +94713068779
>>>
>>> http://sanjeewamalalgoda.blogspot.com/
>>>
>>
>>
>>
>> --
>> ...................................................
>> Dharshana Kasun Warusavitharana
>>
>>
>>
>>
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
--
Sameera Jayasoma
blog: http://sameera.adahas.org
twitter: https://twitter.com/sameerajayasoma
flickr: http://www.flickr.com/photos/sameera-jayasoma/