You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Akila Ravihansa Perera <ra...@wso2.com> on 2015/11/09 04:35:44 UTC
Re: [Dev] Publishing Complete Topology event on Demand
Hi all,
I took the liberty of implementing topology init optimization since that
will speed of lot of things that we're trying to do :)
Here is my approach;
- Introduce a new topic called "initialization" topic
- Introduce a new event called "CompleteTopologyRequestEvent"
- Create a message receiver, processor, listener for this event
- In CC component, create a message receiver object and listen to this
CompleteTopologyRequestEvent
- Once an event of this type is received, publish the CompleteTopologyEvent
*** Update the TopologyReceiver class to publish this
CompleteTopologyRequestEvent so that once you start listening to topology
events, the underlying messaging component will initialize the topology in
an on-demand fashion.
I've created a PR [1] for this and still doing further testing. Please go
through it and let me know your thoughts. I'm planning on writing an
integration test to verify the functionality. I ran a complete build and
all test cases are passing.
Also Stratos startup time improved very noticeably since other components
will not wait for CC to publish the CompleteTopology but rather it will be
requested on-demand via messaging component. Once this is stable I'll move
on to updating Python agent to use it.
[1] https://github.com/apache/stratos/pull/487
Thanks.
On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pu...@wso2.com>
wrote:
> +1 for this approach. I hope PCA plugins will be able to use this.
>
> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com> wrote:
>
>> +1
>>
>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <re...@wso2.com>
>> wrote:
>>
>>> +1. Great thought. It would increase the startup time of the components
>>> those who listens to CompleteTopology and PCA.
>>>
>>> Thanks,
>>> Reka
>>>
>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Hi Devs,
>>>>
>>>> I would like to propose $subject.
>>>>
>>>> Currently we do this with a one minute interval and its causing the
>>>> cartridge agent and the servers to wait for around two minutes (in the
>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>
>>>> I think we can implement $subject by introducing another topic (say
>>>> topology-init) and asking the clients to request Complete Topology event
>>>> via the new topic. This can be done in messaging component without
>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>> accordingly. WDYT?
>>>>
>>>> Thanks
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Reka Thirunavukkarasu
>>> Senior Software Engineer,
>>> WSO2, Inc.:http://wso2.com,
>>> Mobile: +94776442007
>>>
>>>
>>>
>>
>>
>> --
>>
>> Udara Liyanage
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean. enterprise. middleware
>>
>> web: http://udaraliyanage.wordpress.com
>> phone: +94 71 443 6897
>>
>
>
>
> --
>
> *Pubudu Gunatilaka*
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean.enterprise.middleware
> mobile: +94 77 4078049
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: [Dev] Publishing Complete Topology event on Demand
Posted by Isuru Haththotuwa <is...@apache.org>.
Great work Akila! This is definitely a tremendous improvement.
On Wed, Nov 11, 2015 at 3:27 PM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:
> Hi,
>
> I've implemented the following improvements and merged in [1]. JIRAs
> created [2, 3, 4] to track these tasks.
>
> - On-demand initialization of complete topology
> - On-demand initialization of complete application topology
> - On-demand initialization of complete tenant model
> - On-demand initialization of complete application signups
> - New MB topic named "initializer" and new events to request the above
> - Updated PCA to use request events to initialize topology and tenant
> model
>
> - Updated Stratos integration test to verify initialization of complete
> topology, application model, tenant model and application signups within
> 20s. I've added logs in the test case to print actual time it took to
> initialize those. On average it takes less than 1s now.
>
> - Updated Python agent live test case to verify topology and tenant
> models are requested via request events published to initializer topic and
> in-memory models are initialized properly.
>
> [1]
> https://github.com/apache/stratos/commit/60b801144dcb05e3664353c412bc1c3fffd7c55c
> [2] https://issues.apache.org/jira/browse/STRATOS-1544
> [3] https://issues.apache.org/jira/browse/STRATOS-1612
> [4] https://issues.apache.org/jira/browse/STRATOS-1611
>
> Thanks.
>
> On Tue, Nov 10, 2015 at 9:03 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>>
>>
>> On Tue, Nov 10, 2015 at 8:42 AM, Akila Ravihansa Perera <
>> ravihansa@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> I've updated Python agent to use this method to initialize the topology.
>>> Also extended the same logic to initialize complete application model.
>>>
>>> @Imesh: Sure, will do a thorough test before merging.
>>>
>>> @Isuru: I think we cannot get rid of complete topology synchronization
>>> task (periodically published event), since there can be situations where
>>> complete topology request event could be missed due to unavailability of CC
>>> component (maintenance restarts, failures etc.).
>>>
>> +1. For now we can increase the time gap between two complete topology
>> events to stop flooding the network unnecessarily. WDYT?
>>
>>>
>>> Also currently Python agent does not have a mechanism to build the
>>> topology on-the-fly using topology events. It relies on complete topology
>>> event to update its topology context. Perhaps we can think this through
>>> more carefully and remove the periodical publish task in the next major
>>> release.
>>>
>>> Thanks.
>>>
>>> On Tue, Nov 10, 2015 at 6:07 AM, Isuru Haththotuwa <is...@apache.org>
>>> wrote:
>>>
>>>> Great work Akila. Just to clarify, are we removing the Complete
>>>> Topology event and its usages?
>>>>
>>>> On Tue, Nov 10, 2015 at 4:16 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Great work Akila!
>>>>>
>>>>> Please do a thorough test and make sure everything is covered before
>>>>> pushing this change to the current code base. If not I would like to
>>>>> suggest to push this to the next major release.
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera <
>>>>> ravihansa@wso2.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I took the liberty of implementing topology init optimization since
>>>>>> that will speed of lot of things that we're trying to do :)
>>>>>>
>>>>>> Here is my approach;
>>>>>>
>>>>>> - Introduce a new topic called "initialization" topic
>>>>>> - Introduce a new event called "CompleteTopologyRequestEvent"
>>>>>> - Create a message receiver, processor, listener for this event
>>>>>> - In CC component, create a message receiver object and listen to
>>>>>> this CompleteTopologyRequestEvent
>>>>>> - Once an event of this type is received, publish the
>>>>>> CompleteTopologyEvent
>>>>>>
>>>>>> *** Update the TopologyReceiver class to publish this
>>>>>> CompleteTopologyRequestEvent so that once you start listening to topology
>>>>>> events, the underlying messaging component will initialize the topology in
>>>>>> an on-demand fashion.
>>>>>>
>>>>>> I've created a PR [1] for this and still doing further testing.
>>>>>> Please go through it and let me know your thoughts. I'm planning on writing
>>>>>> an integration test to verify the functionality. I ran a complete build and
>>>>>> all test cases are passing.
>>>>>>
>>>>>> Also Stratos startup time improved very noticeably since other
>>>>>> components will not wait for CC to publish the CompleteTopology but rather
>>>>>> it will be requested on-demand via messaging component. Once this is stable
>>>>>> I'll move on to updating Python agent to use it.
>>>>>>
>>>>>> [1] https://github.com/apache/stratos/pull/487
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pubudug@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> +1 for this approach. I hope PCA plugins will be able to use this.
>>>>>>>
>>>>>>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <
>>>>>>>> reka@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> +1. Great thought. It would increase the startup time of the
>>>>>>>>> components those who listens to CompleteTopology and PCA.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Reka
>>>>>>>>>
>>>>>>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <imesh@apache.org
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Hi Devs,
>>>>>>>>>>
>>>>>>>>>> I would like to propose $subject.
>>>>>>>>>>
>>>>>>>>>> Currently we do this with a one minute interval and its causing
>>>>>>>>>> the cartridge agent and the servers to wait for around two minutes (in the
>>>>>>>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>>>>>>>
>>>>>>>>>> I think we can implement $subject by introducing another topic
>>>>>>>>>> (say topology-init) and asking the clients to request Complete Topology
>>>>>>>>>> event via the new topic. This can be done in messaging component without
>>>>>>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>>>>>>>> accordingly. WDYT?
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>
>>>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Reka Thirunavukkarasu
>>>>>>>>> Senior Software Engineer,
>>>>>>>>> WSO2, Inc.:http://wso2.com,
>>>>>>>>> Mobile: +94776442007
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Udara Liyanage
>>>>>>>> Software Engineer
>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>> lean. enterprise. middleware
>>>>>>>>
>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>> phone: +94 71 443 6897
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Pubudu Gunatilaka*
>>>>>>> Software Engineer
>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>> lean.enterprise.middleware
>>>>>>> mobile: +94 77 4078049
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Akila Ravihansa Perera
>>>>>> WSO2 Inc.; http://wso2.com/
>>>>>>
>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Senior Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>>
>>>>> Isuru H.
>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>
>>>>>
>>>>> * <http://wso2.com/>*
>>>>>
>>>>>
>>>>>
>>>
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.; http://wso2.com/
>>>
>>> Blog: http://ravihansa3000.blogspot.com
>>>
>>> --
>>> <http://ravihansa3000.blogspot.com>
>>> <http://ravihansa3000.blogspot.com>
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> <http://ravihansa3000.blogspot.com>
>>> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: [Dev] Publishing Complete Topology event on Demand
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi,
I've implemented the following improvements and merged in [1]. JIRAs
created [2, 3, 4] to track these tasks.
- On-demand initialization of complete topology
- On-demand initialization of complete application topology
- On-demand initialization of complete tenant model
- On-demand initialization of complete application signups
- New MB topic named "initializer" and new events to request the above
- Updated PCA to use request events to initialize topology and tenant model
- Updated Stratos integration test to verify initialization of complete
topology, application model, tenant model and application signups within
20s. I've added logs in the test case to print actual time it took to
initialize those. On average it takes less than 1s now.
- Updated Python agent live test case to verify topology and tenant models
are requested via request events published to initializer topic and
in-memory models are initialized properly.
[1]
https://github.com/apache/stratos/commit/60b801144dcb05e3664353c412bc1c3fffd7c55c
[2] https://issues.apache.org/jira/browse/STRATOS-1544
[3] https://issues.apache.org/jira/browse/STRATOS-1612
[4] https://issues.apache.org/jira/browse/STRATOS-1611
Thanks.
On Tue, Nov 10, 2015 at 9:03 AM, Isuru Haththotuwa <is...@apache.org>
wrote:
>
>
> On Tue, Nov 10, 2015 at 8:42 AM, Akila Ravihansa Perera <
> ravihansa@wso2.com> wrote:
>
>> Hi,
>>
>> I've updated Python agent to use this method to initialize the topology.
>> Also extended the same logic to initialize complete application model.
>>
>> @Imesh: Sure, will do a thorough test before merging.
>>
>> @Isuru: I think we cannot get rid of complete topology synchronization
>> task (periodically published event), since there can be situations where
>> complete topology request event could be missed due to unavailability of CC
>> component (maintenance restarts, failures etc.).
>>
> +1. For now we can increase the time gap between two complete topology
> events to stop flooding the network unnecessarily. WDYT?
>
>>
>> Also currently Python agent does not have a mechanism to build the
>> topology on-the-fly using topology events. It relies on complete topology
>> event to update its topology context. Perhaps we can think this through
>> more carefully and remove the periodical publish task in the next major
>> release.
>>
>> Thanks.
>>
>> On Tue, Nov 10, 2015 at 6:07 AM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>>> Great work Akila. Just to clarify, are we removing the Complete Topology
>>> event and its usages?
>>>
>>> On Tue, Nov 10, 2015 at 4:16 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Great work Akila!
>>>>
>>>> Please do a thorough test and make sure everything is covered before
>>>> pushing this change to the current code base. If not I would like to
>>>> suggest to push this to the next major release.
>>>>
>>>> Thanks
>>>>
>>>> On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera <
>>>> ravihansa@wso2.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I took the liberty of implementing topology init optimization since
>>>>> that will speed of lot of things that we're trying to do :)
>>>>>
>>>>> Here is my approach;
>>>>>
>>>>> - Introduce a new topic called "initialization" topic
>>>>> - Introduce a new event called "CompleteTopologyRequestEvent"
>>>>> - Create a message receiver, processor, listener for this event
>>>>> - In CC component, create a message receiver object and listen to this
>>>>> CompleteTopologyRequestEvent
>>>>> - Once an event of this type is received, publish the
>>>>> CompleteTopologyEvent
>>>>>
>>>>> *** Update the TopologyReceiver class to publish this
>>>>> CompleteTopologyRequestEvent so that once you start listening to topology
>>>>> events, the underlying messaging component will initialize the topology in
>>>>> an on-demand fashion.
>>>>>
>>>>> I've created a PR [1] for this and still doing further testing. Please
>>>>> go through it and let me know your thoughts. I'm planning on writing an
>>>>> integration test to verify the functionality. I ran a complete build and
>>>>> all test cases are passing.
>>>>>
>>>>> Also Stratos startup time improved very noticeably since other
>>>>> components will not wait for CC to publish the CompleteTopology but rather
>>>>> it will be requested on-demand via messaging component. Once this is stable
>>>>> I'll move on to updating Python agent to use it.
>>>>>
>>>>> [1] https://github.com/apache/stratos/pull/487
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pu...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> +1 for this approach. I hope PCA plugins will be able to use this.
>>>>>>
>>>>>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <
>>>>>>> reka@wso2.com> wrote:
>>>>>>>
>>>>>>>> +1. Great thought. It would increase the startup time of the
>>>>>>>> components those who listens to CompleteTopology and PCA.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Reka
>>>>>>>>
>>>>>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Devs,
>>>>>>>>>
>>>>>>>>> I would like to propose $subject.
>>>>>>>>>
>>>>>>>>> Currently we do this with a one minute interval and its causing
>>>>>>>>> the cartridge agent and the servers to wait for around two minutes (in the
>>>>>>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>>>>>>
>>>>>>>>> I think we can implement $subject by introducing another topic
>>>>>>>>> (say topology-init) and asking the clients to request Complete Topology
>>>>>>>>> event via the new topic. This can be done in messaging component without
>>>>>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>>>>>>> accordingly. WDYT?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Imesh Gunaratne
>>>>>>>>>
>>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Reka Thirunavukkarasu
>>>>>>>> Senior Software Engineer,
>>>>>>>> WSO2, Inc.:http://wso2.com,
>>>>>>>> Mobile: +94776442007
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Udara Liyanage
>>>>>>> Software Engineer
>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>> lean. enterprise. middleware
>>>>>>>
>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>> phone: +94 71 443 6897
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Pubudu Gunatilaka*
>>>>>> Software Engineer
>>>>>> WSO2, Inc.: http://wso2.com
>>>>>> lean.enterprise.middleware
>>>>>> mobile: +94 77 4078049
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Akila Ravihansa Perera
>>>>> WSO2 Inc.; http://wso2.com/
>>>>>
>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048* <http://wso2.com/>*
>>>>
>>>>
>>>> * <http://wso2.com/>*
>>>>
>>>>
>>>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>>
>> --
>> <http://ravihansa3000.blogspot.com>
>> <http://ravihansa3000.blogspot.com>
>> Thanks and Regards,
>>
>> Isuru H.
>> <http://ravihansa3000.blogspot.com>
>> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: [Dev] Publishing Complete Topology event on Demand
Posted by Isuru Haththotuwa <is...@apache.org>.
On Tue, Nov 10, 2015 at 8:42 AM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:
> Hi,
>
> I've updated Python agent to use this method to initialize the topology.
> Also extended the same logic to initialize complete application model.
>
> @Imesh: Sure, will do a thorough test before merging.
>
> @Isuru: I think we cannot get rid of complete topology synchronization
> task (periodically published event), since there can be situations where
> complete topology request event could be missed due to unavailability of CC
> component (maintenance restarts, failures etc.).
>
+1. For now we can increase the time gap between two complete topology
events to stop flooding the network unnecessarily. WDYT?
>
> Also currently Python agent does not have a mechanism to build the
> topology on-the-fly using topology events. It relies on complete topology
> event to update its topology context. Perhaps we can think this through
> more carefully and remove the periodical publish task in the next major
> release.
>
> Thanks.
>
> On Tue, Nov 10, 2015 at 6:07 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>> Great work Akila. Just to clarify, are we removing the Complete Topology
>> event and its usages?
>>
>> On Tue, Nov 10, 2015 at 4:16 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Great work Akila!
>>>
>>> Please do a thorough test and make sure everything is covered before
>>> pushing this change to the current code base. If not I would like to
>>> suggest to push this to the next major release.
>>>
>>> Thanks
>>>
>>> On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera <
>>> ravihansa@wso2.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I took the liberty of implementing topology init optimization since
>>>> that will speed of lot of things that we're trying to do :)
>>>>
>>>> Here is my approach;
>>>>
>>>> - Introduce a new topic called "initialization" topic
>>>> - Introduce a new event called "CompleteTopologyRequestEvent"
>>>> - Create a message receiver, processor, listener for this event
>>>> - In CC component, create a message receiver object and listen to this
>>>> CompleteTopologyRequestEvent
>>>> - Once an event of this type is received, publish the
>>>> CompleteTopologyEvent
>>>>
>>>> *** Update the TopologyReceiver class to publish this
>>>> CompleteTopologyRequestEvent so that once you start listening to topology
>>>> events, the underlying messaging component will initialize the topology in
>>>> an on-demand fashion.
>>>>
>>>> I've created a PR [1] for this and still doing further testing. Please
>>>> go through it and let me know your thoughts. I'm planning on writing an
>>>> integration test to verify the functionality. I ran a complete build and
>>>> all test cases are passing.
>>>>
>>>> Also Stratos startup time improved very noticeably since other
>>>> components will not wait for CC to publish the CompleteTopology but rather
>>>> it will be requested on-demand via messaging component. Once this is stable
>>>> I'll move on to updating Python agent to use it.
>>>>
>>>> [1] https://github.com/apache/stratos/pull/487
>>>>
>>>> Thanks.
>>>>
>>>> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pu...@wso2.com>
>>>> wrote:
>>>>
>>>>> +1 for this approach. I hope PCA plugins will be able to use this.
>>>>>
>>>>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <
>>>>>> reka@wso2.com> wrote:
>>>>>>
>>>>>>> +1. Great thought. It would increase the startup time of the
>>>>>>> components those who listens to CompleteTopology and PCA.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Reka
>>>>>>>
>>>>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Devs,
>>>>>>>>
>>>>>>>> I would like to propose $subject.
>>>>>>>>
>>>>>>>> Currently we do this with a one minute interval and its causing the
>>>>>>>> cartridge agent and the servers to wait for around two minutes (in the
>>>>>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>>>>>
>>>>>>>> I think we can implement $subject by introducing another topic (say
>>>>>>>> topology-init) and asking the clients to request Complete Topology event
>>>>>>>> via the new topic. This can be done in messaging component without
>>>>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>>>>>> accordingly. WDYT?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Reka Thirunavukkarasu
>>>>>>> Senior Software Engineer,
>>>>>>> WSO2, Inc.:http://wso2.com,
>>>>>>> Mobile: +94776442007
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Udara Liyanage
>>>>>> Software Engineer
>>>>>> WSO2, Inc.: http://wso2.com
>>>>>> lean. enterprise. middleware
>>>>>>
>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>> phone: +94 71 443 6897
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Pubudu Gunatilaka*
>>>>> Software Engineer
>>>>> WSO2, Inc.: http://wso2.com
>>>>> lean.enterprise.middleware
>>>>> mobile: +94 77 4078049
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Akila Ravihansa Perera
>>>> WSO2 Inc.; http://wso2.com/
>>>>
>>>> Blog: http://ravihansa3000.blogspot.com
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Senior Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
> --
> <http://ravihansa3000.blogspot.com>
> <http://ravihansa3000.blogspot.com>
> Thanks and Regards,
>
> Isuru H.
> <http://ravihansa3000.blogspot.com>
> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: [Dev] Publishing Complete Topology event on Demand
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi,
I've updated Python agent to use this method to initialize the topology.
Also extended the same logic to initialize complete application model.
@Imesh: Sure, will do a thorough test before merging.
@Isuru: I think we cannot get rid of complete topology synchronization task
(periodically published event), since there can be situations where
complete topology request event could be missed due to unavailability of CC
component (maintenance restarts, failures etc.).
Also currently Python agent does not have a mechanism to build the topology
on-the-fly using topology events. It relies on complete topology event to
update its topology context. Perhaps we can think this through more
carefully and remove the periodical publish task in the next major release.
Thanks.
On Tue, Nov 10, 2015 at 6:07 AM, Isuru Haththotuwa <is...@apache.org>
wrote:
> Great work Akila. Just to clarify, are we removing the Complete Topology
> event and its usages?
>
> On Tue, Nov 10, 2015 at 4:16 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> Great work Akila!
>>
>> Please do a thorough test and make sure everything is covered before
>> pushing this change to the current code base. If not I would like to
>> suggest to push this to the next major release.
>>
>> Thanks
>>
>> On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera <
>> ravihansa@wso2.com> wrote:
>>
>>> Hi all,
>>>
>>> I took the liberty of implementing topology init optimization since that
>>> will speed of lot of things that we're trying to do :)
>>>
>>> Here is my approach;
>>>
>>> - Introduce a new topic called "initialization" topic
>>> - Introduce a new event called "CompleteTopologyRequestEvent"
>>> - Create a message receiver, processor, listener for this event
>>> - In CC component, create a message receiver object and listen to this
>>> CompleteTopologyRequestEvent
>>> - Once an event of this type is received, publish the
>>> CompleteTopologyEvent
>>>
>>> *** Update the TopologyReceiver class to publish this
>>> CompleteTopologyRequestEvent so that once you start listening to topology
>>> events, the underlying messaging component will initialize the topology in
>>> an on-demand fashion.
>>>
>>> I've created a PR [1] for this and still doing further testing. Please
>>> go through it and let me know your thoughts. I'm planning on writing an
>>> integration test to verify the functionality. I ran a complete build and
>>> all test cases are passing.
>>>
>>> Also Stratos startup time improved very noticeably since other
>>> components will not wait for CC to publish the CompleteTopology but rather
>>> it will be requested on-demand via messaging component. Once this is stable
>>> I'll move on to updating Python agent to use it.
>>>
>>> [1] https://github.com/apache/stratos/pull/487
>>>
>>> Thanks.
>>>
>>> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pu...@wso2.com>
>>> wrote:
>>>
>>>> +1 for this approach. I hope PCA plugins will be able to use this.
>>>>
>>>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <reka@wso2.com
>>>>> > wrote:
>>>>>
>>>>>> +1. Great thought. It would increase the startup time of the
>>>>>> components those who listens to CompleteTopology and PCA.
>>>>>>
>>>>>> Thanks,
>>>>>> Reka
>>>>>>
>>>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Devs,
>>>>>>>
>>>>>>> I would like to propose $subject.
>>>>>>>
>>>>>>> Currently we do this with a one minute interval and its causing the
>>>>>>> cartridge agent and the servers to wait for around two minutes (in the
>>>>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>>>>
>>>>>>> I think we can implement $subject by introducing another topic (say
>>>>>>> topology-init) and asking the clients to request Complete Topology event
>>>>>>> via the new topic. This can be done in messaging component without
>>>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>>>>> accordingly. WDYT?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Senior Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Reka Thirunavukkarasu
>>>>>> Senior Software Engineer,
>>>>>> WSO2, Inc.:http://wso2.com,
>>>>>> Mobile: +94776442007
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Udara Liyanage
>>>>> Software Engineer
>>>>> WSO2, Inc.: http://wso2.com
>>>>> lean. enterprise. middleware
>>>>>
>>>>> web: http://udaraliyanage.wordpress.com
>>>>> phone: +94 71 443 6897
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Pubudu Gunatilaka*
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean.enterprise.middleware
>>>> mobile: +94 77 4078049
>>>>
>>>
>>>
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.; http://wso2.com/
>>>
>>> Blog: http://ravihansa3000.blogspot.com
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: [Dev] Publishing Complete Topology event on Demand
Posted by Isuru Haththotuwa <is...@apache.org>.
Great work Akila. Just to clarify, are we removing the Complete Topology
event and its usages?
On Tue, Nov 10, 2015 at 4:16 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Great work Akila!
>
> Please do a thorough test and make sure everything is covered before
> pushing this change to the current code base. If not I would like to
> suggest to push this to the next major release.
>
> Thanks
>
> On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera <
> ravihansa@wso2.com> wrote:
>
>> Hi all,
>>
>> I took the liberty of implementing topology init optimization since that
>> will speed of lot of things that we're trying to do :)
>>
>> Here is my approach;
>>
>> - Introduce a new topic called "initialization" topic
>> - Introduce a new event called "CompleteTopologyRequestEvent"
>> - Create a message receiver, processor, listener for this event
>> - In CC component, create a message receiver object and listen to this
>> CompleteTopologyRequestEvent
>> - Once an event of this type is received, publish the
>> CompleteTopologyEvent
>>
>> *** Update the TopologyReceiver class to publish this
>> CompleteTopologyRequestEvent so that once you start listening to topology
>> events, the underlying messaging component will initialize the topology in
>> an on-demand fashion.
>>
>> I've created a PR [1] for this and still doing further testing. Please go
>> through it and let me know your thoughts. I'm planning on writing an
>> integration test to verify the functionality. I ran a complete build and
>> all test cases are passing.
>>
>> Also Stratos startup time improved very noticeably since other components
>> will not wait for CC to publish the CompleteTopology but rather it will be
>> requested on-demand via messaging component. Once this is stable I'll move
>> on to updating Python agent to use it.
>>
>> [1] https://github.com/apache/stratos/pull/487
>>
>> Thanks.
>>
>> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pu...@wso2.com>
>> wrote:
>>
>>> +1 for this approach. I hope PCA plugins will be able to use this.
>>>
>>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com> wrote:
>>>
>>>> +1
>>>>
>>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <re...@wso2.com>
>>>> wrote:
>>>>
>>>>> +1. Great thought. It would increase the startup time of the
>>>>> components those who listens to CompleteTopology and PCA.
>>>>>
>>>>> Thanks,
>>>>> Reka
>>>>>
>>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Devs,
>>>>>>
>>>>>> I would like to propose $subject.
>>>>>>
>>>>>> Currently we do this with a one minute interval and its causing the
>>>>>> cartridge agent and the servers to wait for around two minutes (in the
>>>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>>>
>>>>>> I think we can implement $subject by introducing another topic (say
>>>>>> topology-init) and asking the clients to request Complete Topology event
>>>>>> via the new topic. This can be done in messaging component without
>>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>>>> accordingly. WDYT?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Senior Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Reka Thirunavukkarasu
>>>>> Senior Software Engineer,
>>>>> WSO2, Inc.:http://wso2.com,
>>>>> Mobile: +94776442007
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Udara Liyanage
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean. enterprise. middleware
>>>>
>>>> web: http://udaraliyanage.wordpress.com
>>>> phone: +94 71 443 6897
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> *Pubudu Gunatilaka*
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean.enterprise.middleware
>>> mobile: +94 77 4078049
>>>
>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>>
>
>
>
> --
> Imesh Gunaratne
>
> Senior Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: [Dev] Publishing Complete Topology event on Demand
Posted by Imesh Gunaratne <im...@apache.org>.
Great work Akila!
Please do a thorough test and make sure everything is covered before
pushing this change to the current code base. If not I would like to
suggest to push this to the next major release.
Thanks
On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:
> Hi all,
>
> I took the liberty of implementing topology init optimization since that
> will speed of lot of things that we're trying to do :)
>
> Here is my approach;
>
> - Introduce a new topic called "initialization" topic
> - Introduce a new event called "CompleteTopologyRequestEvent"
> - Create a message receiver, processor, listener for this event
> - In CC component, create a message receiver object and listen to this
> CompleteTopologyRequestEvent
> - Once an event of this type is received, publish the CompleteTopologyEvent
>
> *** Update the TopologyReceiver class to publish this
> CompleteTopologyRequestEvent so that once you start listening to topology
> events, the underlying messaging component will initialize the topology in
> an on-demand fashion.
>
> I've created a PR [1] for this and still doing further testing. Please go
> through it and let me know your thoughts. I'm planning on writing an
> integration test to verify the functionality. I ran a complete build and
> all test cases are passing.
>
> Also Stratos startup time improved very noticeably since other components
> will not wait for CC to publish the CompleteTopology but rather it will be
> requested on-demand via messaging component. Once this is stable I'll move
> on to updating Python agent to use it.
>
> [1] https://github.com/apache/stratos/pull/487
>
> Thanks.
>
> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pu...@wso2.com>
> wrote:
>
>> +1 for this approach. I hope PCA plugins will be able to use this.
>>
>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> +1
>>>
>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu <re...@wso2.com>
>>> wrote:
>>>
>>>> +1. Great thought. It would increase the startup time of the components
>>>> those who listens to CompleteTopology and PCA.
>>>>
>>>> Thanks,
>>>> Reka
>>>>
>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> I would like to propose $subject.
>>>>>
>>>>> Currently we do this with a one minute interval and its causing the
>>>>> cartridge agent and the servers to wait for around two minutes (in the
>>>>> worst case) to start. In Docker this is a considerable amount of delay.
>>>>>
>>>>> I think we can implement $subject by introducing another topic (say
>>>>> topology-init) and asking the clients to request Complete Topology event
>>>>> via the new topic. This can be done in messaging component without
>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be updated
>>>>> accordingly. WDYT?
>>>>>
>>>>> Thanks
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Senior Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Reka Thirunavukkarasu
>>>> Senior Software Engineer,
>>>> WSO2, Inc.:http://wso2.com,
>>>> Mobile: +94776442007
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>>
>> --
>>
>> *Pubudu Gunatilaka*
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean.enterprise.middleware
>> mobile: +94 77 4078049
>>
>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
--
Imesh Gunaratne
Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos