You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Tharindu Munasinghe <th...@cse.mrt.ac.lk> on 2016/01/28 09:29:24 UTC
Problem with init() Method of Window Processors in Apache Stratos CEP Extension
Hi all,
We have written a custom window processor [1] following the same code
structure as in [2], and invoked it via [3], as part of a project for
improving Apache Stratos prediction techniques. Previously we had assumed
that window processors are instantiated one-time when the corresponding
execution plan is deployed, so we thought we can maintain the history of
health stat information inside the window processors.
However, when testing our implementation on AWS infrastructure with an
external CEP instance (v3.1.0) we noticed that the init() method of window
processors attached to the same stream is invoked multiple times. It
appears that the invocation happens periodically. As a result, the Stratos
autoscaler receives multiple health stat events of the same type within a
given time interval. We investigated the problem with an existing window
processor [2] and observed the same phenomenon.
At which stage(s) could the init() method be getting invoked, under our
current implementation? Also, it is possible that multiple window processor
instances could be getting instantiated in case of exceptions occurring in
the window processor?
[1]
https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/stratos-cep-extension/wso2cep-3.1.0/src/main/java/org/apache/stratos/cep/extension/PredictorWindowProcessor.java
[2]
https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/stratos-cep-extension/wso2cep-3.1.0/src/main/java/org/apache/stratos/cep/extension/GradientFinderWindowProcessor.java
[3]
https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/artifacts/wso2cep-3.1.0/executionplans/PredictorOfHealthRequest.xml
Thanks,
Tharindu
--
*Th**a**rindu Munasin**ghe.*
*Undergraduate ,Department of Computer **S**cience and Engineering*
*University of Moratuwa.*
*Contact no. +94770460887*
Re: Problem with init() Method of Window Processors in Apache Stratos
CEP Extension
Posted by Tharindu Munasinghe <mu...@gmail.com>.
Hi Mohan,
Thanks a lot for the information. We saw some strange behavior that
repeats the creation of window processors. According to your
explanation, it should be due to some other reason. we will check and
update.
Thanks,
Tharindu.
Tharindu Munasinghe.
Undergraduate ,Department of Computer Science and Engineering
University of Moratuwa.
Contact no. +94770460887
On Thu, Jan 28, 2016 at 4:39 PM, Mohanadarshan Vivekanandalingam
<mo...@wso2.com> wrote:
>
>
> On Thu, Jan 28, 2016 at 1:59 PM, Tharindu Munasinghe
> <th...@cse.mrt.ac.lk> wrote:
>>
>> Hi all,
>>
>> We have written a custom window processor [1] following the same code
>> structure as in [2], and invoked it via [3], as part of a project for
>> improving Apache Stratos prediction techniques. Previously we had assumed
>> that window processors are instantiated one-time when the corresponding
>> execution plan is deployed, so we thought we can maintain the history of
>> health stat information inside the window processors.
>>
>> However, when testing our implementation on AWS infrastructure with an
>> external CEP instance (v3.1.0) we noticed that the init() method of window
>> processors attached to the same stream is invoked multiple times. It appears
>> that the invocation happens periodically. As a result, the Stratos
>> autoscaler receives multiple health stat events of the same type within a
>> given time interval. We investigated the problem with an existing window
>> processor [2] and observed the same phenomenon.
>>
>> At which stage(s) could the init() method be getting invoked, under our
>> current implementation? Also, it is possible that multiple window processor
>> instances could be getting instantiated in case of exceptions occurring in
>> the window processor?
>
>
> Yes, It is a known behavior of Siddhi where init method is called twice (not
> many times).. AFAIR it called in validation flow and actual flow.. In latest
> Siddhi this was implemented differently by introducing new functions which
> called only once..
>
> Thanks,
> Mohan
>
>>
>>
>>
>> [1]
>> https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/stratos-cep-extension/wso2cep-3.1.0/src/main/java/org/apache/stratos/cep/extension/PredictorWindowProcessor.java
>>
>> [2]
>> https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/stratos-cep-extension/wso2cep-3.1.0/src/main/java/org/apache/stratos/cep/extension/GradientFinderWindowProcessor.java
>>
>> [3]
>> https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/artifacts/wso2cep-3.1.0/executionplans/PredictorOfHealthRequest.xml
>>
>> Thanks,
>> Tharindu
>> --
>> Tharindu Munasinghe.
>> Undergraduate ,Department of Computer Science and Engineering
>> University of Moratuwa.
>> Contact no. +94770460887
>
>
>
>
> --
> V. Mohanadarshan
> Senior Software Engineer,
> Data Technologies Team,
> WSO2, Inc. http://wso2.com
> lean.enterprise.middleware.
>
> email: mohan@wso2.com
> phone:(+94) 771117673
Re: Problem with init() Method of Window Processors in Apache Stratos
CEP Extension
Posted by Mohanadarshan Vivekanandalingam <mo...@wso2.com>.
On Thu, Jan 28, 2016 at 1:59 PM, Tharindu Munasinghe <
tharindu.11@cse.mrt.ac.lk> wrote:
> Hi all,
>
> We have written a custom window processor [1] following the same code
> structure as in [2], and invoked it via [3], as part of a project for
> improving Apache Stratos prediction techniques. Previously we had assumed
> that window processors are instantiated one-time when the corresponding
> execution plan is deployed, so we thought we can maintain the history of
> health stat information inside the window processors.
>
> However, when testing our implementation on AWS infrastructure with an
> external CEP instance (v3.1.0) we noticed that the init() method of window
> processors attached to the same stream is invoked multiple times. It
> appears that the invocation happens periodically. As a result, the Stratos
> autoscaler receives multiple health stat events of the same type within a
> given time interval. We investigated the problem with an existing window
> processor [2] and observed the same phenomenon.
>
> At which stage(s) could the init() method be getting invoked, under our
> current implementation? Also, it is possible that multiple window processor
> instances could be getting instantiated in case of exceptions occurring in
> the window processor?
>
Yes, It is a known behavior of Siddhi where init method is called twice
(not many times).. AFAIR it called in validation flow and actual flow.. In
latest Siddhi this was implemented differently by introducing new functions
which called only once..
Thanks,
Mohan
>
>
> [1]
> https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/stratos-cep-extension/wso2cep-3.1.0/src/main/java/org/apache/stratos/cep/extension/PredictorWindowProcessor.java
>
> [2]
> https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/stratos-cep-extension/wso2cep-3.1.0/src/main/java/org/apache/stratos/cep/extension/GradientFinderWindowProcessor.java
>
> [3]
> https://github.com/ridwannaruto/stratos/blob/fyp/extensions/cep/modules/artifacts/wso2cep-3.1.0/executionplans/PredictorOfHealthRequest.xml
>
> Thanks,
> Tharindu
> --
> *Th**a**rindu Munasin**ghe.*
> *Undergraduate ,Department of Computer **S**cience and Engineering*
> *University of Moratuwa.*
> *Contact no. +94770460887*
>
--
*V. Mohanadarshan*
*Senior Software Engineer,*
*Data Technologies Team,*
*WSO2, Inc. http://wso2.com <http://wso2.com> *
*lean.enterprise.middleware.*
email: mohan@wso2.com
phone:(+94) 771117673