You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Nirmal Fernando <ni...@gmail.com> on 2013/10/20 20:16:46 UTC

Load Balancer and CEP integration

Hi All,

I've successfully integrated Stratos LB and WSO2 CEP (selected CEP engine).

Load balancer and CEP is communicating via Observer pattern, where a
statistics collector in LB acting as the subject and CEPs act as Observers.
Stats collector periodically notifies all the registered observers, so that
they can act upon changes. I'm committing the changes soon.

Now, when I load Stratos LB, CEP emits periodical aggregated statistics on
requests in flight at LB level, to a topic.

Following are few aggregated stats emitted by CEP:

{"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
{"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
{"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}

{"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
{"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
{"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}



-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Load Balancer and CEP integration

Posted by Imesh Gunaratne <im...@apache.org>.
Great work Nirmal!


On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <ni...@gmail.com>wrote:

> Hi All,
>
> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
> engine).
>
> Load balancer and CEP is communicating via Observer pattern, where a
> statistics collector in LB acting as the subject and CEPs act as Observers.
> Stats collector periodically notifies all the registered observers, so that
> they can act upon changes. I'm committing the changes soon.
>
> Now, when I load Stratos LB, CEP emits periodical aggregated statistics on
> requests in flight at LB level, to a topic.
>
> Following are few aggregated stats emitted by CEP:
>
>
> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>
>
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>

Re: Load Balancer and CEP integration

Posted by John Mathon <jo...@wso2.com>.
+1 


On Oct 20, 2013, at 11:50 AM, Nirmal Fernando <ni...@gmail.com> wrote:

> 
> 
> 
> On Sun, Oct 20, 2013 at 11:51 PM, Sanjiva Weerawarana <sa...@wso2.com> wrote:
> Awesome!
> 
> We'll have to be very careful about the memory usage of the stat collectors ..
> 
> +1. I'm already on it. So far it behaves nicely with load. 
> 
> <Screenshot from 2013-10-21 00:18:57.png> 
> I remember that in previous occasions those beasts ended up being trouble makers.
> 
> Sanjiva.
> 
> 
> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <ni...@gmail.com> wrote:
> Hi All,
> 
> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP engine). 
> 
> Load balancer and CEP is communicating via Observer pattern, where a statistics collector in LB acting as the subject and CEPs act as Observers. Stats collector periodically notifies all the registered observers, so that they can act upon changes. I'm committing the changes soon.
> 
> Now, when I load Stratos LB, CEP emits periodical aggregated statistics on requests in flight at LB level, to a topic.
> 
> Following are few aggregated stats emitted by CEP:
> 
> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
> 
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
> 
> 
> 
> -- 
> Best Regards,
> Nirmal
> 
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
> 
> Blog: http://nirmalfdo.blogspot.com/
> 
> 
> 
> -- 
> Sanjiva Weerawarana, Ph.D.
> Founder, Chairman & CEO; WSO2, Inc.;  http://wso2.com/
> email: sanjiva@wso2.com; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1 650 265 8311
> blog: http://sanjiva.weerawarana.org/
> 
> Lean . Enterprise . Middleware
> 
> 
> 
> -- 
> Best Regards,
> Nirmal
> 
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
> 
> Blog: http://nirmalfdo.blogspot.com/

Re: Load Balancer and CEP integration

Posted by Nirmal Fernando <ni...@gmail.com>.
On Sun, Oct 20, 2013 at 11:51 PM, Sanjiva Weerawarana <sa...@wso2.com>wrote:

> Awesome!
>
> We'll have to be very careful about the memory usage of the stat
> collectors ..
>

+1. I'm already on it. So far it behaves nicely with load.

[image: Inline image 1]

> I remember that in previous occasions those beasts ended up being trouble
> makers.
>
> Sanjiva.
>
>
> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <ni...@gmail.com>wrote:
>
>> Hi All,
>>
>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>> engine).
>>
>> Load balancer and CEP is communicating via Observer pattern, where a
>> statistics collector in LB acting as the subject and CEPs act as Observers.
>> Stats collector periodically notifies all the registered observers, so that
>> they can act upon changes. I'm committing the changes soon.
>>
>> Now, when I load Stratos LB, CEP emits periodical aggregated statistics
>> on requests in flight at LB level, to a topic.
>>
>> Following are few aggregated stats emitted by CEP:
>>
>>
>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>
>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>
>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>
>>
>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>
>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
>
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, Chairman & CEO; WSO2, Inc.;  http://wso2.com/
> email: sanjiva@wso2.com; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1
> 650 265 8311
> blog: http://sanjiva.weerawarana.org/
>
> Lean . Enterprise . Middleware
>



-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Load Balancer and CEP integration

Posted by Sanjiva Weerawarana <sa...@wso2.com>.
Awesome!

We'll have to be very careful about the memory usage of the stat collectors
.. I remember that in previous occasions those beasts ended up being
trouble makers.

Sanjiva.


On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <ni...@gmail.com>wrote:

> Hi All,
>
> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
> engine).
>
> Load balancer and CEP is communicating via Observer pattern, where a
> statistics collector in LB acting as the subject and CEPs act as Observers.
> Stats collector periodically notifies all the registered observers, so that
> they can act upon changes. I'm committing the changes soon.
>
> Now, when I load Stratos LB, CEP emits periodical aggregated statistics on
> requests in flight at LB level, to a topic.
>
> Following are few aggregated stats emitted by CEP:
>
>
> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>
>
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>



-- 
Sanjiva Weerawarana, Ph.D.
Founder, Chairman & CEO; WSO2, Inc.;  http://wso2.com/
email: sanjiva@wso2.com; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1
650 265 8311
blog: http://sanjiva.weerawarana.org/

Lean . Enterprise . Middleware

Re: Load Balancer and CEP integration

Posted by Lahiru Sandaruwan <la...@wso2.com>.
On Wed, Oct 23, 2013 at 1:38 PM, Nirmal Fernando <ni...@gmail.com>wrote:

> Lahiru, what I said was to 'build up its own information model, on top of
> topology model.', not to build up everything. So, the idea is you build up
> a model while keeping references to the existing topology model.
>

+1.. We have implemented with an Autoscaler Context which has required
items for Autoscaler.

Thanks.

>
>
> On Wed, Oct 23, 2013 at 12:26 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>
>>
>>
>>
>> On Wed, Oct 23, 2013 at 12:11 PM, Nirmal Fernando <nirmal070125@gmail.com
>> > wrote:
>>
>>> Lahiru,
>>>
>>> IMO autoscaler needs to build up its own information model, on top of
>>> topology model. This is applicable for all Stratos components.
>>>
>>
>> It is a overhead to have a separate model for few parameters. Topology is
>> currently used by LB and Autoscaler. It already has details which is
>> specific to LB.
>>
>>  Also in autoscaler side, it has parameters specific for clusters as well
>> as members. So if we implement a separate information model in Autoscaler
>> side, it has to override all the topology message processors to detect
>> topology events.
>>
>> So the users of topology can ignore the parameters they do not require.
>>
>>>
>>>
>>> On Wed, Oct 23, 2013 at 11:57 AM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>>>
>>>> Hi Nirmal,
>>>>
>>>> Autoscaler needs the service id for finding cluster. Otherwise it has
>>>> to traverse through all services to find it. Please add service id to the
>>>> event.
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On Mon, Oct 21, 2013 at 1:50 PM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Oct 21, 2013 at 1:25 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>>>>>
>>>>>> Hi Nirmal,
>>>>>>
>>>>>> Nice work!
>>>>>>
>>>>>> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <
>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>>>>>>> engine).
>>>>>>>
>>>>>>> Load balancer and CEP is communicating via Observer pattern, where a
>>>>>>> statistics collector in LB acting as the subject and CEPs act as Observers.
>>>>>>> Stats collector periodically notifies all the registered observers, so that
>>>>>>> they can act upon changes. I'm committing the changes soon.
>>>>>>>
>>>>>>> Now, when I load Stratos LB, CEP emits periodical aggregated
>>>>>>> statistics on requests in flight at LB level, to a topic.
>>>>>>>
>>>>>>> Following are few aggregated stats emitted by CEP:
>>>>>>>
>>>>>>>
>>>>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>>>>>>
>>>>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>>>>>>
>>>>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>>>>>>
>>>>>>
>>>>>> Hope above three are separate events sent to the topic
>>>>>> "summarized_stat_topic". Autoscaler will get these and keep latest values
>>>>>> against cluster "c1".
>>>>>>
>>>>>
>>>>> Lahiru, yes correct. (summarized-health-stats topic)
>>>>>
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>>>>>>
>>>>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>>>>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Nirmal
>>>>>>>
>>>>>>> Nirmal Fernando.
>>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>>
>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> --
>>>>>> Lahiru Sandaruwan
>>>>>> Software Engineer,
>>>>>> Platform Technologies,
>>>>>> WSO2 Inc., http://wso2.com
>>>>>> lean.enterprise.middleware
>>>>>>
>>>>>> email: lahirus@wso2.com cell: (+94) 773 325 954
>>>>>> blog: http://lahiruwrites.blogspot.com/
>>>>>> twitter: http://twitter.com/lahirus
>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> Nirmal Fernando.
>>>>> PPMC Member & Committer of Apache Stratos,
>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Lahiru Sandaruwan
>>>> Software Engineer,
>>>> Platform Technologies,
>>>> WSO2 Inc., http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> email: lahirus@wso2.com cell: (+94) 773 325 954
>>>> blog: http://lahiruwrites.blogspot.com/
>>>> twitter: http://twitter.com/lahirus
>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Nirmal
>>>
>>> Nirmal Fernando.
>>> PPMC Member & Committer of Apache Stratos,
>>> Senior Software Engineer, WSO2 Inc.
>>>
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>
>>
>>
>> --
>> --
>> Lahiru Sandaruwan
>> Software Engineer,
>> Platform Technologies,
>> WSO2 Inc., http://wso2.com
>> lean.enterprise.middleware
>>
>> email: lahirus@wso2.com cell: (+94) 773 325 954
>> blog: http://lahiruwrites.blogspot.com/
>> twitter: http://twitter.com/lahirus
>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>
>>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>



-- 
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Re: Load Balancer and CEP integration

Posted by Nirmal Fernando <ni...@gmail.com>.
Lahiru, what I said was to 'build up its own information model, on top of
topology model.', not to build up everything. So, the idea is you build up
a model while keeping references to the existing topology model.


On Wed, Oct 23, 2013 at 12:26 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:

>
>
>
> On Wed, Oct 23, 2013 at 12:11 PM, Nirmal Fernando <ni...@gmail.com>wrote:
>
>> Lahiru,
>>
>> IMO autoscaler needs to build up its own information model, on top of
>> topology model. This is applicable for all Stratos components.
>>
>
> It is a overhead to have a separate model for few parameters. Topology is
> currently used by LB and Autoscaler. It already has details which is
> specific to LB.
>
> Also in autoscaler side, it has parameters specific for clusters as well
> as members. So if we implement a separate information model in Autoscaler
> side, it has to override all the topology message processors to detect
> topology events.
>
> So the users of topology can ignore the parameters they do not require.
>
>>
>>
>> On Wed, Oct 23, 2013 at 11:57 AM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>>
>>> Hi Nirmal,
>>>
>>> Autoscaler needs the service id for finding cluster. Otherwise it has to
>>> traverse through all services to find it. Please add service id to the
>>> event.
>>>
>>> Thanks.
>>>
>>>
>>> On Mon, Oct 21, 2013 at 1:50 PM, Nirmal Fernando <nirmal070125@gmail.com
>>> > wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Mon, Oct 21, 2013 at 1:25 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>>>>
>>>>> Hi Nirmal,
>>>>>
>>>>> Nice work!
>>>>>
>>>>> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <
>>>>> nirmal070125@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>>>>>> engine).
>>>>>>
>>>>>> Load balancer and CEP is communicating via Observer pattern, where a
>>>>>> statistics collector in LB acting as the subject and CEPs act as Observers.
>>>>>> Stats collector periodically notifies all the registered observers, so that
>>>>>> they can act upon changes. I'm committing the changes soon.
>>>>>>
>>>>>> Now, when I load Stratos LB, CEP emits periodical aggregated
>>>>>> statistics on requests in flight at LB level, to a topic.
>>>>>>
>>>>>> Following are few aggregated stats emitted by CEP:
>>>>>>
>>>>>>
>>>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>>>>>
>>>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>>>>>
>>>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>>>>>
>>>>>
>>>>> Hope above three are separate events sent to the topic
>>>>> "summarized_stat_topic". Autoscaler will get these and keep latest values
>>>>> against cluster "c1".
>>>>>
>>>>
>>>> Lahiru, yes correct. (summarized-health-stats topic)
>>>>
>>>>>
>>>>> Thanks.
>>>>>
>>>>>>
>>>>>>
>>>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>>>>>
>>>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>>>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Nirmal
>>>>>>
>>>>>> Nirmal Fernando.
>>>>>> PPMC Member & Committer of Apache Stratos,
>>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>>
>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Lahiru Sandaruwan
>>>>> Software Engineer,
>>>>> Platform Technologies,
>>>>> WSO2 Inc., http://wso2.com
>>>>> lean.enterprise.middleware
>>>>>
>>>>> email: lahirus@wso2.com cell: (+94) 773 325 954
>>>>> blog: http://lahiruwrites.blogspot.com/
>>>>> twitter: http://twitter.com/lahirus
>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Nirmal
>>>>
>>>> Nirmal Fernando.
>>>> PPMC Member & Committer of Apache Stratos,
>>>> Senior Software Engineer, WSO2 Inc.
>>>>
>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Lahiru Sandaruwan
>>> Software Engineer,
>>> Platform Technologies,
>>> WSO2 Inc., http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> email: lahirus@wso2.com cell: (+94) 773 325 954
>>> blog: http://lahiruwrites.blogspot.com/
>>> twitter: http://twitter.com/lahirus
>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Software Engineer,
> Platform Technologies,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Load Balancer and CEP integration

Posted by Lahiru Sandaruwan <la...@wso2.com>.
On Wed, Oct 23, 2013 at 12:11 PM, Nirmal Fernando <ni...@gmail.com>wrote:

> Lahiru,
>
> IMO autoscaler needs to build up its own information model, on top of
> topology model. This is applicable for all Stratos components.
>

It is a overhead to have a separate model for few parameters. Topology is
currently used by LB and Autoscaler. It already has details which is
specific to LB.

Also in autoscaler side, it has parameters specific for clusters as well as
members. So if we implement a separate information model in Autoscaler
side, it has to override all the topology message processors to detect
topology events.

So the users of topology can ignore the parameters they do not require.

>
>
> On Wed, Oct 23, 2013 at 11:57 AM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>
>> Hi Nirmal,
>>
>> Autoscaler needs the service id for finding cluster. Otherwise it has to
>> traverse through all services to find it. Please add service id to the
>> event.
>>
>> Thanks.
>>
>>
>> On Mon, Oct 21, 2013 at 1:50 PM, Nirmal Fernando <ni...@gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Mon, Oct 21, 2013 at 1:25 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>>>
>>>> Hi Nirmal,
>>>>
>>>> Nice work!
>>>>
>>>> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>>>>> engine).
>>>>>
>>>>> Load balancer and CEP is communicating via Observer pattern, where a
>>>>> statistics collector in LB acting as the subject and CEPs act as Observers.
>>>>> Stats collector periodically notifies all the registered observers, so that
>>>>> they can act upon changes. I'm committing the changes soon.
>>>>>
>>>>> Now, when I load Stratos LB, CEP emits periodical aggregated
>>>>> statistics on requests in flight at LB level, to a topic.
>>>>>
>>>>> Following are few aggregated stats emitted by CEP:
>>>>>
>>>>>
>>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>>>>
>>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>>>>
>>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>>>>
>>>>
>>>> Hope above three are separate events sent to the topic
>>>> "summarized_stat_topic". Autoscaler will get these and keep latest values
>>>> against cluster "c1".
>>>>
>>>
>>> Lahiru, yes correct. (summarized-health-stats topic)
>>>
>>>>
>>>> Thanks.
>>>>
>>>>>
>>>>>
>>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>>>>
>>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> Nirmal Fernando.
>>>>> PPMC Member & Committer of Apache Stratos,
>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Lahiru Sandaruwan
>>>> Software Engineer,
>>>> Platform Technologies,
>>>> WSO2 Inc., http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> email: lahirus@wso2.com cell: (+94) 773 325 954
>>>> blog: http://lahiruwrites.blogspot.com/
>>>> twitter: http://twitter.com/lahirus
>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Nirmal
>>>
>>> Nirmal Fernando.
>>> PPMC Member & Committer of Apache Stratos,
>>> Senior Software Engineer, WSO2 Inc.
>>>
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>
>>
>>
>> --
>> --
>> Lahiru Sandaruwan
>> Software Engineer,
>> Platform Technologies,
>> WSO2 Inc., http://wso2.com
>> lean.enterprise.middleware
>>
>> email: lahirus@wso2.com cell: (+94) 773 325 954
>> blog: http://lahiruwrites.blogspot.com/
>> twitter: http://twitter.com/lahirus
>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>
>>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>



-- 
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Re: Load Balancer and CEP integration

Posted by Nirmal Fernando <ni...@gmail.com>.
Lahiru,

IMO autoscaler needs to build up its own information model, on top of
topology model. This is applicable for all Stratos components.


On Wed, Oct 23, 2013 at 11:57 AM, Lahiru Sandaruwan <la...@wso2.com>wrote:

> Hi Nirmal,
>
> Autoscaler needs the service id for finding cluster. Otherwise it has to
> traverse through all services to find it. Please add service id to the
> event.
>
> Thanks.
>
>
> On Mon, Oct 21, 2013 at 1:50 PM, Nirmal Fernando <ni...@gmail.com>wrote:
>
>>
>>
>>
>> On Mon, Oct 21, 2013 at 1:25 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>>
>>> Hi Nirmal,
>>>
>>> Nice work!
>>>
>>> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <
>>> nirmal070125@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>>>> engine).
>>>>
>>>> Load balancer and CEP is communicating via Observer pattern, where a
>>>> statistics collector in LB acting as the subject and CEPs act as Observers.
>>>> Stats collector periodically notifies all the registered observers, so that
>>>> they can act upon changes. I'm committing the changes soon.
>>>>
>>>> Now, when I load Stratos LB, CEP emits periodical aggregated statistics
>>>> on requests in flight at LB level, to a topic.
>>>>
>>>> Following are few aggregated stats emitted by CEP:
>>>>
>>>>
>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>>>
>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>>>
>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>>>
>>>
>>> Hope above three are separate events sent to the topic
>>> "summarized_stat_topic". Autoscaler will get these and keep latest values
>>> against cluster "c1".
>>>
>>
>> Lahiru, yes correct. (summarized-health-stats topic)
>>
>>>
>>> Thanks.
>>>
>>>>
>>>>
>>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>>>
>>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Nirmal
>>>>
>>>> Nirmal Fernando.
>>>> PPMC Member & Committer of Apache Stratos,
>>>> Senior Software Engineer, WSO2 Inc.
>>>>
>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Lahiru Sandaruwan
>>> Software Engineer,
>>> Platform Technologies,
>>> WSO2 Inc., http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> email: lahirus@wso2.com cell: (+94) 773 325 954
>>> blog: http://lahiruwrites.blogspot.com/
>>> twitter: http://twitter.com/lahirus
>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Software Engineer,
> Platform Technologies,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Load Balancer and CEP integration

Posted by Lahiru Sandaruwan <la...@wso2.com>.
Hi Nirmal,

Autoscaler needs the service id for finding cluster. Otherwise it has to
traverse through all services to find it. Please add service id to the
event.

Thanks.


On Mon, Oct 21, 2013 at 1:50 PM, Nirmal Fernando <ni...@gmail.com>wrote:

>
>
>
> On Mon, Oct 21, 2013 at 1:25 PM, Lahiru Sandaruwan <la...@wso2.com>wrote:
>
>> Hi Nirmal,
>>
>> Nice work!
>>
>> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <nirmal070125@gmail.com
>> > wrote:
>>
>>> Hi All,
>>>
>>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>>> engine).
>>>
>>> Load balancer and CEP is communicating via Observer pattern, where a
>>> statistics collector in LB acting as the subject and CEPs act as Observers.
>>> Stats collector periodically notifies all the registered observers, so that
>>> they can act upon changes. I'm committing the changes soon.
>>>
>>> Now, when I load Stratos LB, CEP emits periodical aggregated statistics
>>> on requests in flight at LB level, to a topic.
>>>
>>> Following are few aggregated stats emitted by CEP:
>>>
>>>
>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>>
>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>>
>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>>
>>
>> Hope above three are separate events sent to the topic
>> "summarized_stat_topic". Autoscaler will get these and keep latest values
>> against cluster "c1".
>>
>
> Lahiru, yes correct. (summarized-health-stats topic)
>
>>
>> Thanks.
>>
>>>
>>>
>>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>>
>>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Nirmal
>>>
>>> Nirmal Fernando.
>>> PPMC Member & Committer of Apache Stratos,
>>> Senior Software Engineer, WSO2 Inc.
>>>
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>
>>
>>
>> --
>> --
>> Lahiru Sandaruwan
>> Software Engineer,
>> Platform Technologies,
>> WSO2 Inc., http://wso2.com
>> lean.enterprise.middleware
>>
>> email: lahirus@wso2.com cell: (+94) 773 325 954
>> blog: http://lahiruwrites.blogspot.com/
>> twitter: http://twitter.com/lahirus
>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>
>>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>



-- 
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Re: Load Balancer and CEP integration

Posted by Nirmal Fernando <ni...@gmail.com>.
On Mon, Oct 21, 2013 at 1:25 PM, Lahiru Sandaruwan <la...@wso2.com> wrote:

> Hi Nirmal,
>
> Nice work!
>
> On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <ni...@gmail.com>wrote:
>
>> Hi All,
>>
>> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
>> engine).
>>
>> Load balancer and CEP is communicating via Observer pattern, where a
>> statistics collector in LB acting as the subject and CEPs act as Observers.
>> Stats collector periodically notifies all the registered observers, so that
>> they can act upon changes. I'm committing the changes soon.
>>
>> Now, when I load Stratos LB, CEP emits periodical aggregated statistics
>> on requests in flight at LB level, to a topic.
>>
>> Following are few aggregated stats emitted by CEP:
>>
>>
>> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>>
>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>>
>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>>
>
> Hope above three are separate events sent to the topic
> "summarized_stat_topic". Autoscaler will get these and keep latest values
> against cluster "c1".
>

Lahiru, yes correct. (summarized-health-stats topic)

>
> Thanks.
>
>>
>>
>> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>>
>> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
>> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Software Engineer,
> Platform Technologies,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Load Balancer and CEP integration

Posted by Lahiru Sandaruwan <la...@wso2.com>.
Hi Nirmal,

Nice work!

On Sun, Oct 20, 2013 at 11:46 PM, Nirmal Fernando <ni...@gmail.com>wrote:

> Hi All,
>
> I've successfully integrated Stratos LB and WSO2 CEP (selected CEP
> engine).
>
> Load balancer and CEP is communicating via Observer pattern, where a
> statistics collector in LB acting as the subject and CEPs act as Observers.
> Stats collector periodically notifies all the registered observers, so that
> they can act upon changes. I'm committing the changes soon.
>
> Now, when I load Stratos LB, CEP emits periodical aggregated statistics on
> requests in flight at LB level, to a topic.
>
> Following are few aggregated stats emitted by CEP:
>
>
> {"average_requests_in_flight":{"cluster_id":"c1","value":"78.66666666666667"}}
>
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"0.17776829680194836"}}
>
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.0666637038353851"}}
>

Hope above three are separate events sent to the topic
"summarized_stat_topic". Autoscaler will get these and keep latest values
against cluster "c1".

Thanks.

>
>
> {"second_derivative_of_requests_in_flight":{"cluster_id":"c1","value":"-0.004444888953094816"}}
>
> {"gradient_of_requests_in_flight":{"cluster_id":"c1","value":"0.04444246922359006"}}
> {"average_requests_in_flight":{"cluster_id":"c1","value":"9.5"}}
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>



-- 
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146