You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Darren Shepherd <da...@gmail.com> on 2013/10/03 00:48:46 UTC
[MERGE] spring-modularization to master - Spring Modularization
Not sure how this works... I would like to merge in the new
modularized Spring setup to master. There is info on the wiki about it
[1] [2] [3]. The primary change is to break apart the monolithic
applicationContext.xml and componentContext.xml files such that each
plugin can maintain and contribute its own configuration.
In addition to breaking up the configuration we no longer use ACS
custom AOP and it is now fully Spring AOP.
Now adding/removing a plug-in is a matter of just adding a jar to the
classpath (exception being commands.properties, I'll address that in a
different thread). Unfortunately this branch does not have the
changes to package things in different RPMs. So it would be great if
somebody could take up the packaging effort to split out all the
plugins into different RPMs.
Darren
[1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring
[2] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-ins%2C+Modules%2C+and+Extensions
[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
Sure can switch jenkins.buildacloud.org to build off of
spring-modularization? It will at least fix the DB upgrade issues. I
really don't understand the whole test environment. What tests are
ran where, how to get "BVT" running etc.
Darren
On Thu, Oct 3, 2013 at 7:58 AM, Chip Childers <ch...@sungard.com> wrote:
> On Wed, Oct 2, 2013 at 8:59 PM, Darren Shepherd <darren.s.shepherd@gmail.com
>> wrote:
>
>> It's actually really easy to turn this off. If you look at the last
>> commit of the branch you'll see what changes to turn it on. So you can
>> just reverse that one commit to disable it.
>>
>> I'd rather merge this and then do the BVTs on master if possible. I
>> really want to get this committed. Every time somebody changes a spring
>> config file I have to keep making the corresponding changes. It seems like
>> people are gearing up and developing stuff right now for 4.3. So the
>> longer we wait the more headache it is for everyone.
>>
>> Again, if we find out this is a complete disaster, I can just flip some
>> config settings and were back to the old style. I had to keep
>> compatibility with the old style as Awsapi and usage still use it.
>>
>> Darren
>>
>
>
> I get the desire, but *please* let's use BVT on this. We, unfortunately,
> have a poor track record with changes like this breaking things in master.
> I'm not saying this as a judgement / measurement of your branch, I'm
> making a general historical statement. We really need to hold ourselves to
> the goal of keeping master stable through clear test results in feature
> branches prior to a merge.
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Chip Childers <ch...@sungard.com>.
On Wed, Oct 2, 2013 at 8:59 PM, Darren Shepherd <darren.s.shepherd@gmail.com
> wrote:
> It's actually really easy to turn this off. If you look at the last
> commit of the branch you'll see what changes to turn it on. So you can
> just reverse that one commit to disable it.
>
> I'd rather merge this and then do the BVTs on master if possible. I
> really want to get this committed. Every time somebody changes a spring
> config file I have to keep making the corresponding changes. It seems like
> people are gearing up and developing stuff right now for 4.3. So the
> longer we wait the more headache it is for everyone.
>
> Again, if we find out this is a complete disaster, I can just flip some
> config settings and were back to the old style. I had to keep
> compatibility with the old style as Awsapi and usage still use it.
>
> Darren
>
I get the desire, but *please* let's use BVT on this. We, unfortunately,
have a poor track record with changes like this breaking things in master.
I'm not saying this as a judgement / measurement of your branch, I'm
making a general historical statement. We really need to hold ourselves to
the goal of keeping master stable through clear test results in feature
branches prior to a merge.
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
It's actually really easy to turn this off. If you look at the last commit of the branch you'll see what changes to turn it on. So you can just reverse that one commit to disable it.
I'd rather merge this and then do the BVTs on master if possible. I really want to get this committed. Every time somebody changes a spring config file I have to keep making the corresponding changes. It seems like people are gearing up and developing stuff right now for 4.3. So the longer we wait the more headache it is for everyone.
Again, if we find out this is a complete disaster, I can just flip some config settings and were back to the old style. I had to keep compatibility with the old style as Awsapi and usage still use it.
Darren
> On Oct 2, 2013, at 5:42 PM, Darren Shepherd <da...@gmail.com> wrote:
>
> Yes agreed. I've extensively tested this, but that is never enough. How do I get the BVTs ran against this. Due to the cross cutting nature of this I want to get this merged as fast as possible.
>
> Darren
>
>> On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
>>
>> +1 on running the BVT on it. We've been through this one once before. Should be careful.
>>
>> --Alex
>>
>>> -----Original Message-----
>>> From: Kelven Yang [mailto:kelven.yang@citrix.com]
>>> Sent: Wednesday, October 2, 2013 4:39 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [MERGE] spring-modularization to master - Spring
>>> Modularization
>>>
>>> Darren,
>>>
>>> This looks really nice. A few questions on Spring AOP replacement.
>>>
>>> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>>> AOP is mainly due to that inside existing CloudStack codebase, we have many
>>> places that are doing run-time type-casting, the code in these places
>>> assumes a real object that implements all interfaces in the semantics context.
>>> At the time when I initially converted to Spring, I couldn't ensure that
>>> switching to proxy-based AOP can have 100% coverage for these run-time
>>> cases. What is your approach to address this run-time type-casting issue?
>>>
>>> 2) We've run into a huge-memory footprint issue that may be caused by
>>> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
>>> layer. Do you have a chance to run a memory analysis in the heap after
>>> management server is started.
>>>
>>> I might be good to run BVT test on the branch before the merge, could
>>> someone initiate the effort?
>>>
>>> kelven
>>>
>>>
>>>
>>> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
>>> wrote:
>>>
>>>> Not sure how this works... I would like to merge in the new
>>>> modularized Spring setup to master. There is info on the wiki about it
>>>> [1] [2] [3]. The primary change is to break apart the monolithic
>>>> applicationContext.xml and componentContext.xml files such that each
>>>> plugin can maintain and contribute its own configuration.
>>>>
>>>> In addition to breaking up the configuration we no longer use ACS
>>>> custom AOP and it is now fully Spring AOP.
>>>>
>>>> Now adding/removing a plug-in is a matter of just adding a jar to the
>>>> classpath (exception being commands.properties, I'll address that in a
>>>> different thread). Unfortunately this branch does not have the changes
>>>> to package things in different RPMs. So it would be great if somebody
>>>> could take up the packaging effort to split out all the plugins into
>>>> different RPMs.
>>>>
>>>> Darren
>>>>
>>>> [1]
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
>>> rin
>>>> g
>>>> [2]
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
>>> ins%2C+Modu
>>>> les
>>>> %2C+and+Extensions
>>>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Prasanna Santhanam <ts...@apache.org>.
On Thu, Oct 03, 2013 at 02:46:40PM +0000, Rayees Namathponnan wrote:
> Hi Prasanna,
>
> If we are running BVT or regression on specific branch, marvin
> should also from same branch right ? but we always getting marvin
> from master branch job "cloudstack-marvin"
>
Yes, but marvin's core is the same in master and 4.2 at the moment.
That will change if and when the merge of the refactoring happens. Can
you help fix the jenkins job to fetch marvin from the repo the test is
targetted towards?
--
Prasanna.,
------------------------
Powered by BigRock.com
RE: [MERGE] spring-modularization to master - Spring Modularization
Posted by Rayees Namathponnan <ra...@citrix.com>.
Hi Prasanna,
If we are running BVT or regression on specific branch, marvin should also from same branch right ? but we always getting marvin from master branch job "cloudstack-marvin"
Regards,
Rayees
-----Original Message-----
From: Prasanna Santhanam [mailto:tsp@apache.org]
Sent: Wednesday, October 02, 2013 10:06 PM
To: dev@cloudstack.apache.org
Subject: Re: [MERGE] spring-modularization to master - Spring Modularization
I switched the test infrastructure on jenkins.buildacloud.org to run the bvts [1] against master last week. Couple of weeks before that the simulator [2] tests were switched to run against master. Both are broken unfortunately and the bvt and checkin tests aren't running.
I've filed a bug here:
https://issues.apache.org/jira//browse/CLOUDSTACK-4791<https://issues.apache.org/jira/browse/CLOUDSTACK-4791>
[1] http://jenkins.buildacloud.org/view/cloudstack-qa/
[2] http://jenkins.buildacloud.org/view/simulator/
Anyone who has access to the jenkins can run the bvts on their desired branch. Simply login and change the test-yumrepo-refresh job to point to your branch. Build that to refresh the remote repository with the packages made from your branch. Then switch test-matrix to point to the same development branch and fire a build. That's about it.
On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> Yes agreed. I've extensively tested this, but that is never enough.
> How do I get the BVTs ran against this. Due to the cross cutting
> nature of this I want to get this merged as fast as possible.
>
> Darren
>
> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com>> wrote:
> >
> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
> >
> > --Alex
> >
> >> -----Original Message-----
> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
> >> Sent: Wednesday, October 2, 2013 4:39 PM
> >> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> >> Modularization
> >>
> >> Darren,
> >>
> >> This looks really nice. A few questions on Spring AOP replacement.
> >>
> >> 1) Spring AOP is proxy-based, the reason we ended up of using
> >> customized AOP is mainly due to that inside existing CloudStack
> >> codebase, we have many places that are doing run-time type-casting,
> >> the code in these places assumes a real object that implements all interfaces in the semantics context.
> >> At the time when I initially converted to Spring, I couldn't ensure
> >> that switching to proxy-based AOP can have 100% coverage for these
> >> run-time cases. What is your approach to address this run-time type-casting issue?
> >>
> >> 2) We've run into a huge-memory footprint issue that may be caused
> >> by conflicts of CGLIB usage in Spring AOP and the CGLIB usage in
> >> CloudStack Dao layer. Do you have a chance to run a memory analysis
> >> in the heap after management server is started.
> >>
> >> I might be good to run BVT test on the branch before the merge,
> >> could someone initiate the effort?
> >>
> >> kelven
> >>
> >>
> >>
> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>>
> >> wrote:
> >>
> >>> Not sure how this works... I would like to merge in the new
> >>> modularized Spring setup to master. There is info on the wiki
> >>> about it [1] [2] [3]. The primary change is to break apart the
> >>> monolithic applicationContext.xml and componentContext.xml files
> >>> such that each plugin can maintain and contribute its own configuration.
> >>>
> >>> In addition to breaking up the configuration we no longer use ACS
> >>> custom AOP and it is now fully Spring AOP.
> >>>
> >>> Now adding/removing a plug-in is a matter of just adding a jar to
> >>> the classpath (exception being commands.properties, I'll address
> >>> that in a different thread). Unfortunately this branch does not
> >>> have the changes to package things in different RPMs. So it would
> >>> be great if somebody could take up the packaging effort to split
> >>> out all the plugins into different RPMs.
> >>>
> >>> Darren
> >>>
> >>> [1]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+
> >>> Sp
> >> rin
> >>> g
> >>> [2]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >> ins%2C+Modu
> >>> les
> >>> %2C+and+Extensions
> >>> [3]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> >
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Kelven Yang <ke...@citrix.com>.
＋1
Kelven
On 10/9/13 9:18 AM, "Chip Childers" <ch...@sungard.com> wrote:
>+1 from me too.
>
>
>On Wed, Oct 9, 2013 at 12:13 PM, Prasanna Santhanam <ts...@apache.org>
>wrote:
>
>> Here's the BVT result from spring-modularization branch: Failures that
>> are listed exist on master as well. So the branch doesn't break
>> anything additionally.
>>
>> +1 to merge
>>
>> http://jenkins.buildacloud.org/job/test-matrix/571/
>>
>> Xen: Test Run: #892
>> Logs:
>>
>>
>>http://jenkins.buildacloud.org/job/test-matrix/571/distro=centos63,hyperv
>>isor=xen,profile=xen62/
>> --------
>> Total:85
>> Fail :22
>> Skip :1
>> --------
>>
>> name pass fail skip
>> test_internal_lb/ 1 0 0
>> test_iso/ 1 1 0
>> test_pvlan/ 1 0 0
>> test_deploy_vm/ 1 0 0
>> test_volumes/ 0 2 0
>> test_deploy_vms_with_varied_deploymentplanners/ 3 0 0
>> test_guest_vlan_range/ 1 0 0
>> test_regions/ 1 0 0
>> test_vm_life_cycle/ 8 2 0
>> test_non_contigiousvlan/ 1 0 0
>> test_ssvm/ 9 1 0
>> test_disk_offerings/ 3 0 0
>> test_reset_vm_on_reboot/ 1 0 0
>> test_network_acl/ 1 0 0
>> test_global_settings/ 0 2 0
>> test_privategw_acl/ 0 1 0
>> test_affinity_groups/ 0 1 0
>> test_multipleips_per_nic/ 1 0 0
>> test_deploy_vm_with_userdata/ 2 0 0
>> test_loadbalance/ 0 3 0
>> test_templates/ 5 2 1
>> test_public_ip_range/ 1 0 0
>> test_vm_snapshots/ 0 3 0
>> test_resource_detail/ 0 1 0
>> test_portable_publicip/ 2 0 0
>> test_network/ 5 2 0
>> test_routers/ 9 0 0
>> test_nic/ 1 0 0
>> test_scale_vm/ 1 0 0
>> test_service_offerings/ 3 1 0
>> --------
>>
>>
>> Regressions
>> --------
>> name
>> duration age
>> integration.smoke.test_ssvm.TestSSVMs.test_07_reboot_ssvm
>> 86.497 1
>>
>>integration.smoke.test_templates.TestCreateTemplate.test_01_create_templa
>>te
>> 61.837 1
>>
>> Failures
>> --------
>> name
>> duration age
>> <nose.suite.ContextSuite context=TestISO>:setup
>> 128.276 1
>> integration.smoke.test_volumes.TestCreateVolume.test_01_create_volume
>> 0.083 22
>> <nose.suite.ContextSuite context=TestVolumes>:setup
>> 263.343 21
>> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_09_expunge_vm
>> 125.244 21
>>
>>integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDet
>>ach_iso
>> 1059.25 2
>>
>>integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_Upd
>>ateConfigParamWithScope
>> 0.028 21
>>
>>integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_Upd
>>ateConfigParamWithScope
>> 0.059 21
>> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl
>> 208.673 21
>> <nose.suite.ContextSuite context=TestDeployVmWithAffinityGroup>:setup
>> 0 2
>>
>>integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule
>>_src_nat
>> 1023.78 3
>>
>>integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule
>>_non_nat
>> 1003.45 3
>>
>>integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_remova
>>l_lb
>> 1003.46 3
>> integration.smoke.test_templates.TestTemplates.test_03_delete_template
>> 5.12 21
>>
>>integration.smoke.test_vm_snapshots.TestVmSnapshot.test_01_create_vm_snap
>>shots
>> 993.25 2
>>
>>integration.smoke.test_vm_snapshots.TestVmSnapshot.test_02_revert_vm_snap
>>shots
>> 993.239 2
>>
>>integration.smoke.test_vm_snapshots.TestVmSnapshot.test_03_delete_vm_snap
>>shots
>> 1.23 2
>> <nose.suite.ContextSuite context=TestResourceDetail>:setup
>> 0 21
>>
>>integration.smoke.test_network.TestPortForwarding.test_01_port_fwd_on_src
>>_nat
>> 1003.63 2
>>
>>integration.smoke.test_network.TestPortForwarding.test_02_port_fwd_on_non
>>_src_nat
>> 1008.55 2
>>
>>integration.smoke.test_service_offerings.TestServiceOfferings.test_04_cha
>>nge_offering_small
>> 1008.6 2
>>
>> Fixed
>> --------
>> name
>> duration age
>> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_08_migrate_vm
>> 40.891 0
>> integration.smoke.test_scale_vm.TestScaleVm.test_01_scale_vm
>> 5.16 0
>>
>>
>> KVM: Test Run: #891
>> Logs:
>>
>>
>>http://jenkins.buildacloud.org/job/test-matrix/571/distro=centos63,hyperv
>>isor=kvm,profile=kvm-centos63/
>> --------
>> Total:89
>> Fail :22
>> Skip :2
>> --------
>>
>> name pass fail skip
>> test_internal_lb/ 1 0 0
>> test_iso/ 4 1 1
>> test_pvlan/ 1 0 0
>> test_deploy_vm/ 1 0 0
>> test_volumes/ 0 2 0
>> test_deploy_vms_with_varied_deploymentplanners/ 3 0 0
>> test_guest_vlan_range/ 1 0 0
>> test_regions/ 1 0 0
>> test_vm_life_cycle/ 7 3 0
>> test_non_contigiousvlan/ 1 0 0
>> test_ssvm/ 10 0 0
>> test_disk_offerings/ 3 0 0
>> test_reset_vm_on_reboot/ 1 0 0
>> test_network_acl/ 1 0 0
>> test_global_settings/ 0 2 0
>> test_privategw_acl/ 0 1 0
>> test_affinity_groups/ 0 1 0
>> test_multipleips_per_nic/ 1 0 0
>> test_deploy_vm_with_userdata/ 2 0 0
>> test_loadbalance/ 0 3 0
>> test_templates/ 6 1 1
>> test_public_ip_range/ 1 0 0
>> test_vm_snapshots/ 0 3 0
>> test_resource_detail/ 0 1 0
>> test_portable_publicip/ 2 0 0
>> test_network/ 5 2 0
>> test_routers/ 9 0 0
>> test_nic/ 1 0 0
>> test_scale_vm/ 0 1 0
>> test_service_offerings/ 3 1 0
>> --------
>>
>>
>> Regressions
>> --------
>> name
>> duration age
>> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_08_migrate_vm
>> 30.83 1
>>
>>integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDet
>>ach_iso
>> 1058.62 1
>>
>>integration.smoke.test_vm_snapshots.TestVmSnapshot.test_01_create_vm_snap
>>shots
>> 993.179 1
>>
>>integration.smoke.test_vm_snapshots.TestVmSnapshot.test_02_revert_vm_snap
>>shots
>> 993.166 1
>>
>>integration.smoke.test_vm_snapshots.TestVmSnapshot.test_03_delete_vm_snap
>>shots
>> 0.264 1
>>
>>integration.smoke.test_network.TestPortForwarding.test_01_port_fwd_on_src
>>_nat
>> 1003.5 1
>>
>>integration.smoke.test_network.TestPortForwarding.test_02_port_fwd_on_non
>>_src_nat
>> 1008.59 1
>> integration.smoke.test_scale_vm.TestScaleVm.test_01_scale_vm
>> 5.097 1
>>
>>integration.smoke.test_service_offerings.TestServiceOfferings.test_04_cha
>>nge_offering_small
>> 1124.17 1
>>
>> Failures
>> --------
>> name
>> duration age
>> integration.smoke.test_iso.TestISO.test_03_delete_iso
>> 65.215 1
>> integration.smoke.test_volumes.TestCreateVolume.test_01_create_volume
>> 0.082 21
>> <nose.suite.ContextSuite context=TestVolumes>:setup
>> 212.865 20
>> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_09_expunge_vm
>> 125.24 20
>>
>>integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_Upd
>>ateConfigParamWithScope
>> 0.029 20
>>
>>integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_Upd
>>ateConfigParamWithScope
>> 0.052 20
>> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl
>> 56.245 20
>> <nose.suite.ContextSuite context=TestDeployVmWithAffinityGroup>:setup
>> 0 1
>>
>>integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule
>>_src_nat
>> 1003.58 2
>>
>>integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule
>>_non_nat
>> 1003.45 2
>>
>>integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_remova
>>l_lb
>> 1003.48 2
>> integration.smoke.test_templates.TestTemplates.test_03_delete_template
>> 5.12 20
>> <nose.suite.ContextSuite context=TestResourceDetail>:setup
>> 0 20
>>
>> Fixed
>> --------
>> name
>> duration age
>>
>>integration.smoke.test_templates.TestCreateTemplate.test_01_create_templa
>>te
>> 35.415 0
>>
>>
>>
>> On Wed, Oct 09, 2013 at 06:16:04PM +0530, Prasanna Santhanam wrote:
>> > On Tue, Oct 08, 2013 at 10:20:01AM -0700, Darren Shepherd wrote:
>> > > >From what I can gather it seems that master currently fails the BVT
>> > > (and know when I say BVT I mean that black box that apparently
>>exists
>> > > somewhere doing something, but I have no clue what it really means).
>> > > So in turn my spring modularization branch will additionally fail
>>BVT.
>> > > Citrix internal QA ran some tests against my branch and they mostly
>> > > passed but some failed. Its quite difficult to sort through this
>>all
>> > > because tests are failing on master. So I don't know what to do at
>> > > this point. At least my branch won't completely blow up everything.
>> > > I just know the longer it takes to merge this the more painful it
>>will
>> > > be....
>> >
>> > Darren, Sorry about the frustrations. I haven't been able to keep
>> > track these last few weeks about your work.
>> >
>> > I have run the tests against your branch and everything looks good so
>> > far. Whatever is failing is failing on master as well, so we'll ignore
>> > that. I will share the report shortly.
>> >
>> > They are the result of the job here, which I will summarize in a more
>> > readable format and mail out later tonight once it completes:
>> > http://jenkins.buildacloud.org/job/test-matrix/571/
>> >
>> > Each bubble in that grid represents a hypervisor profile against which
>> > all the tests were run.
>> >
>> > > Honestly this is all quite frustrating for myself being new to
>> > > contributing to ACS. I feel somewhat lost in the whole process of
>>how
>> > > to get features in. I'll refrain from venting my frustrations.
>> > >
>> > > Darren
>> >
>> > --
>> > Prasanna.,
>> >
>> > ------------------------
>> > Powered by BigRock.com
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>>
>>
>>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Chip Childers <ch...@sungard.com>.
+1 from me too.
On Wed, Oct 9, 2013 at 12:13 PM, Prasanna Santhanam <ts...@apache.org> wrote:
> Here's the BVT result from spring-modularization branch: Failures that
> are listed exist on master as well. So the branch doesn't break
> anything additionally.
>
> +1 to merge
>
> http://jenkins.buildacloud.org/job/test-matrix/571/
>
> Xen: Test Run: #892
> Logs:
>
> http://jenkins.buildacloud.org/job/test-matrix/571/distro=centos63,hypervisor=xen,profile=xen62/
> --------
> Total:85
> Fail :22
> Skip :1
> --------
>
> name pass fail skip
> test_internal_lb/ 1 0 0
> test_iso/ 1 1 0
> test_pvlan/ 1 0 0
> test_deploy_vm/ 1 0 0
> test_volumes/ 0 2 0
> test_deploy_vms_with_varied_deploymentplanners/ 3 0 0
> test_guest_vlan_range/ 1 0 0
> test_regions/ 1 0 0
> test_vm_life_cycle/ 8 2 0
> test_non_contigiousvlan/ 1 0 0
> test_ssvm/ 9 1 0
> test_disk_offerings/ 3 0 0
> test_reset_vm_on_reboot/ 1 0 0
> test_network_acl/ 1 0 0
> test_global_settings/ 0 2 0
> test_privategw_acl/ 0 1 0
> test_affinity_groups/ 0 1 0
> test_multipleips_per_nic/ 1 0 0
> test_deploy_vm_with_userdata/ 2 0 0
> test_loadbalance/ 0 3 0
> test_templates/ 5 2 1
> test_public_ip_range/ 1 0 0
> test_vm_snapshots/ 0 3 0
> test_resource_detail/ 0 1 0
> test_portable_publicip/ 2 0 0
> test_network/ 5 2 0
> test_routers/ 9 0 0
> test_nic/ 1 0 0
> test_scale_vm/ 1 0 0
> test_service_offerings/ 3 1 0
> --------
>
>
> Regressions
> --------
> name
> duration age
> integration.smoke.test_ssvm.TestSSVMs.test_07_reboot_ssvm
> 86.497 1
> integration.smoke.test_templates.TestCreateTemplate.test_01_create_template
> 61.837 1
>
> Failures
> --------
> name
> duration age
> <nose.suite.ContextSuite context=TestISO>:setup
> 128.276 1
> integration.smoke.test_volumes.TestCreateVolume.test_01_create_volume
> 0.083 22
> <nose.suite.ContextSuite context=TestVolumes>:setup
> 263.343 21
> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_09_expunge_vm
> 125.244 21
> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDetach_iso
> 1059.25 2
> integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope
> 0.028 21
> integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope
> 0.059 21
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl
> 208.673 21
> <nose.suite.ContextSuite context=TestDeployVmWithAffinityGroup>:setup
> 0 2
> integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule_src_nat
> 1023.78 3
> integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule_non_nat
> 1003.45 3
> integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_removal_lb
> 1003.46 3
> integration.smoke.test_templates.TestTemplates.test_03_delete_template
> 5.12 21
> integration.smoke.test_vm_snapshots.TestVmSnapshot.test_01_create_vm_snapshots
> 993.25 2
> integration.smoke.test_vm_snapshots.TestVmSnapshot.test_02_revert_vm_snapshots
> 993.239 2
> integration.smoke.test_vm_snapshots.TestVmSnapshot.test_03_delete_vm_snapshots
> 1.23 2
> <nose.suite.ContextSuite context=TestResourceDetail>:setup
> 0 21
> integration.smoke.test_network.TestPortForwarding.test_01_port_fwd_on_src_nat
> 1003.63 2
> integration.smoke.test_network.TestPortForwarding.test_02_port_fwd_on_non_src_nat
> 1008.55 2
> integration.smoke.test_service_offerings.TestServiceOfferings.test_04_change_offering_small
> 1008.6 2
>
> Fixed
> --------
> name
> duration age
> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_08_migrate_vm
> 40.891 0
> integration.smoke.test_scale_vm.TestScaleVm.test_01_scale_vm
> 5.16 0
>
>
> KVM: Test Run: #891
> Logs:
>
> http://jenkins.buildacloud.org/job/test-matrix/571/distro=centos63,hypervisor=kvm,profile=kvm-centos63/
> --------
> Total:89
> Fail :22
> Skip :2
> --------
>
> name pass fail skip
> test_internal_lb/ 1 0 0
> test_iso/ 4 1 1
> test_pvlan/ 1 0 0
> test_deploy_vm/ 1 0 0
> test_volumes/ 0 2 0
> test_deploy_vms_with_varied_deploymentplanners/ 3 0 0
> test_guest_vlan_range/ 1 0 0
> test_regions/ 1 0 0
> test_vm_life_cycle/ 7 3 0
> test_non_contigiousvlan/ 1 0 0
> test_ssvm/ 10 0 0
> test_disk_offerings/ 3 0 0
> test_reset_vm_on_reboot/ 1 0 0
> test_network_acl/ 1 0 0
> test_global_settings/ 0 2 0
> test_privategw_acl/ 0 1 0
> test_affinity_groups/ 0 1 0
> test_multipleips_per_nic/ 1 0 0
> test_deploy_vm_with_userdata/ 2 0 0
> test_loadbalance/ 0 3 0
> test_templates/ 6 1 1
> test_public_ip_range/ 1 0 0
> test_vm_snapshots/ 0 3 0
> test_resource_detail/ 0 1 0
> test_portable_publicip/ 2 0 0
> test_network/ 5 2 0
> test_routers/ 9 0 0
> test_nic/ 1 0 0
> test_scale_vm/ 0 1 0
> test_service_offerings/ 3 1 0
> --------
>
>
> Regressions
> --------
> name
> duration age
> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_08_migrate_vm
> 30.83 1
> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDetach_iso
> 1058.62 1
> integration.smoke.test_vm_snapshots.TestVmSnapshot.test_01_create_vm_snapshots
> 993.179 1
> integration.smoke.test_vm_snapshots.TestVmSnapshot.test_02_revert_vm_snapshots
> 993.166 1
> integration.smoke.test_vm_snapshots.TestVmSnapshot.test_03_delete_vm_snapshots
> 0.264 1
> integration.smoke.test_network.TestPortForwarding.test_01_port_fwd_on_src_nat
> 1003.5 1
> integration.smoke.test_network.TestPortForwarding.test_02_port_fwd_on_non_src_nat
> 1008.59 1
> integration.smoke.test_scale_vm.TestScaleVm.test_01_scale_vm
> 5.097 1
> integration.smoke.test_service_offerings.TestServiceOfferings.test_04_change_offering_small
> 1124.17 1
>
> Failures
> --------
> name
> duration age
> integration.smoke.test_iso.TestISO.test_03_delete_iso
> 65.215 1
> integration.smoke.test_volumes.TestCreateVolume.test_01_create_volume
> 0.082 21
> <nose.suite.ContextSuite context=TestVolumes>:setup
> 212.865 20
> integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_09_expunge_vm
> 125.24 20
> integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope
> 0.029 20
> integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope
> 0.052 20
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl
> 56.245 20
> <nose.suite.ContextSuite context=TestDeployVmWithAffinityGroup>:setup
> 0 1
> integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule_src_nat
> 1003.58 2
> integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule_non_nat
> 1003.45 2
> integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_removal_lb
> 1003.48 2
> integration.smoke.test_templates.TestTemplates.test_03_delete_template
> 5.12 20
> <nose.suite.ContextSuite context=TestResourceDetail>:setup
> 0 20
>
> Fixed
> --------
> name
> duration age
> integration.smoke.test_templates.TestCreateTemplate.test_01_create_template
> 35.415 0
>
>
>
> On Wed, Oct 09, 2013 at 06:16:04PM +0530, Prasanna Santhanam wrote:
> > On Tue, Oct 08, 2013 at 10:20:01AM -0700, Darren Shepherd wrote:
> > > >From what I can gather it seems that master currently fails the BVT
> > > (and know when I say BVT I mean that black box that apparently exists
> > > somewhere doing something, but I have no clue what it really means).
> > > So in turn my spring modularization branch will additionally fail BVT.
> > > Citrix internal QA ran some tests against my branch and they mostly
> > > passed but some failed. Its quite difficult to sort through this all
> > > because tests are failing on master. So I don't know what to do at
> > > this point. At least my branch won't completely blow up everything.
> > > I just know the longer it takes to merge this the more painful it will
> > > be....
> >
> > Darren, Sorry about the frustrations. I haven't been able to keep
> > track these last few weeks about your work.
> >
> > I have run the tests against your branch and everything looks good so
> > far. Whatever is failing is failing on master as well, so we'll ignore
> > that. I will share the report shortly.
> >
> > They are the result of the job here, which I will summarize in a more
> > readable format and mail out later tonight once it completes:
> > http://jenkins.buildacloud.org/job/test-matrix/571/
> >
> > Each bubble in that grid represents a hypervisor profile against which
> > all the tests were run.
> >
> > > Honestly this is all quite frustrating for myself being new to
> > > contributing to ACS. I feel somewhat lost in the whole process of how
> > > to get features in. I'll refrain from venting my frustrations.
> > >
> > > Darren
> >
> > --
> > Prasanna.,
> >
> > ------------------------
> > Powered by BigRock.com
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
>
>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Prasanna Santhanam <ts...@apache.org>.
Here's the BVT result from spring-modularization branch: Failures that
are listed exist on master as well. So the branch doesn't break
anything additionally.
+1 to merge
http://jenkins.buildacloud.org/job/test-matrix/571/
Xen: Test Run: #892
Logs:
http://jenkins.buildacloud.org/job/test-matrix/571/distro=centos63,hypervisor=xen,profile=xen62/
--------
Total:85
Fail :22
Skip :1
--------
name pass fail skip
test_internal_lb/ 1 0 0
test_iso/ 1 1 0
test_pvlan/ 1 0 0
test_deploy_vm/ 1 0 0
test_volumes/ 0 2 0
test_deploy_vms_with_varied_deploymentplanners/ 3 0 0
test_guest_vlan_range/ 1 0 0
test_regions/ 1 0 0
test_vm_life_cycle/ 8 2 0
test_non_contigiousvlan/ 1 0 0
test_ssvm/ 9 1 0
test_disk_offerings/ 3 0 0
test_reset_vm_on_reboot/ 1 0 0
test_network_acl/ 1 0 0
test_global_settings/ 0 2 0
test_privategw_acl/ 0 1 0
test_affinity_groups/ 0 1 0
test_multipleips_per_nic/ 1 0 0
test_deploy_vm_with_userdata/ 2 0 0
test_loadbalance/ 0 3 0
test_templates/ 5 2 1
test_public_ip_range/ 1 0 0
test_vm_snapshots/ 0 3 0
test_resource_detail/ 0 1 0
test_portable_publicip/ 2 0 0
test_network/ 5 2 0
test_routers/ 9 0 0
test_nic/ 1 0 0
test_scale_vm/ 1 0 0
test_service_offerings/ 3 1 0
--------
Regressions
--------
name duration age
integration.smoke.test_ssvm.TestSSVMs.test_07_reboot_ssvm 86.497 1
integration.smoke.test_templates.TestCreateTemplate.test_01_create_template 61.837 1
Failures
--------
name duration age
<nose.suite.ContextSuite context=TestISO>:setup 128.276 1
integration.smoke.test_volumes.TestCreateVolume.test_01_create_volume 0.083 22
<nose.suite.ContextSuite context=TestVolumes>:setup 263.343 21
integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_09_expunge_vm 125.244 21
integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDetach_iso 1059.25 2
integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope 0.028 21
integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope 0.059 21
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl 208.673 21
<nose.suite.ContextSuite context=TestDeployVmWithAffinityGroup>:setup 0 2
integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule_src_nat 1023.78 3
integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule_non_nat 1003.45 3
integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_removal_lb 1003.46 3
integration.smoke.test_templates.TestTemplates.test_03_delete_template 5.12 21
integration.smoke.test_vm_snapshots.TestVmSnapshot.test_01_create_vm_snapshots 993.25 2
integration.smoke.test_vm_snapshots.TestVmSnapshot.test_02_revert_vm_snapshots 993.239 2
integration.smoke.test_vm_snapshots.TestVmSnapshot.test_03_delete_vm_snapshots 1.23 2
<nose.suite.ContextSuite context=TestResourceDetail>:setup 0 21
integration.smoke.test_network.TestPortForwarding.test_01_port_fwd_on_src_nat 1003.63 2
integration.smoke.test_network.TestPortForwarding.test_02_port_fwd_on_non_src_nat 1008.55 2
integration.smoke.test_service_offerings.TestServiceOfferings.test_04_change_offering_small 1008.6 2
Fixed
--------
name duration age
integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_08_migrate_vm 40.891 0
integration.smoke.test_scale_vm.TestScaleVm.test_01_scale_vm 5.16 0
KVM: Test Run: #891
Logs:
http://jenkins.buildacloud.org/job/test-matrix/571/distro=centos63,hypervisor=kvm,profile=kvm-centos63/
--------
Total:89
Fail :22
Skip :2
--------
name pass fail skip
test_internal_lb/ 1 0 0
test_iso/ 4 1 1
test_pvlan/ 1 0 0
test_deploy_vm/ 1 0 0
test_volumes/ 0 2 0
test_deploy_vms_with_varied_deploymentplanners/ 3 0 0
test_guest_vlan_range/ 1 0 0
test_regions/ 1 0 0
test_vm_life_cycle/ 7 3 0
test_non_contigiousvlan/ 1 0 0
test_ssvm/ 10 0 0
test_disk_offerings/ 3 0 0
test_reset_vm_on_reboot/ 1 0 0
test_network_acl/ 1 0 0
test_global_settings/ 0 2 0
test_privategw_acl/ 0 1 0
test_affinity_groups/ 0 1 0
test_multipleips_per_nic/ 1 0 0
test_deploy_vm_with_userdata/ 2 0 0
test_loadbalance/ 0 3 0
test_templates/ 6 1 1
test_public_ip_range/ 1 0 0
test_vm_snapshots/ 0 3 0
test_resource_detail/ 0 1 0
test_portable_publicip/ 2 0 0
test_network/ 5 2 0
test_routers/ 9 0 0
test_nic/ 1 0 0
test_scale_vm/ 0 1 0
test_service_offerings/ 3 1 0
--------
Regressions
--------
name duration age
integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_08_migrate_vm 30.83 1
integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDetach_iso 1058.62 1
integration.smoke.test_vm_snapshots.TestVmSnapshot.test_01_create_vm_snapshots 993.179 1
integration.smoke.test_vm_snapshots.TestVmSnapshot.test_02_revert_vm_snapshots 993.166 1
integration.smoke.test_vm_snapshots.TestVmSnapshot.test_03_delete_vm_snapshots 0.264 1
integration.smoke.test_network.TestPortForwarding.test_01_port_fwd_on_src_nat 1003.5 1
integration.smoke.test_network.TestPortForwarding.test_02_port_fwd_on_non_src_nat 1008.59 1
integration.smoke.test_scale_vm.TestScaleVm.test_01_scale_vm 5.097 1
integration.smoke.test_service_offerings.TestServiceOfferings.test_04_change_offering_small 1124.17 1
Failures
--------
name duration age
integration.smoke.test_iso.TestISO.test_03_delete_iso 65.215 1
integration.smoke.test_volumes.TestCreateVolume.test_01_create_volume 0.082 21
<nose.suite.ContextSuite context=TestVolumes>:setup 212.865 20
integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_09_expunge_vm 125.24 20
integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope 0.029 20
integration.smoke.test_global_settings.TestUpdateConfigWithScope.test_UpdateConfigParamWithScope 0.052 20
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl 56.245 20
<nose.suite.ContextSuite context=TestDeployVmWithAffinityGroup>:setup 0 1
integration.smoke.test_loadbalance.TestLoadBalance.test_01_create_lb_rule_src_nat 1003.58 2
integration.smoke.test_loadbalance.TestLoadBalance.test_02_create_lb_rule_non_nat 1003.45 2
integration.smoke.test_loadbalance.TestLoadBalance.test_assign_and_removal_lb 1003.48 2
integration.smoke.test_templates.TestTemplates.test_03_delete_template 5.12 20
<nose.suite.ContextSuite context=TestResourceDetail>:setup 0 20
Fixed
--------
name duration age
integration.smoke.test_templates.TestCreateTemplate.test_01_create_template 35.415 0
On Wed, Oct 09, 2013 at 06:16:04PM +0530, Prasanna Santhanam wrote:
> On Tue, Oct 08, 2013 at 10:20:01AM -0700, Darren Shepherd wrote:
> > >From what I can gather it seems that master currently fails the BVT
> > (and know when I say BVT I mean that black box that apparently exists
> > somewhere doing something, but I have no clue what it really means).
> > So in turn my spring modularization branch will additionally fail BVT.
> > Citrix internal QA ran some tests against my branch and they mostly
> > passed but some failed. Its quite difficult to sort through this all
> > because tests are failing on master. So I don't know what to do at
> > this point. At least my branch won't completely blow up everything.
> > I just know the longer it takes to merge this the more painful it will
> > be....
>
> Darren, Sorry about the frustrations. I haven't been able to keep
> track these last few weeks about your work.
>
> I have run the tests against your branch and everything looks good so
> far. Whatever is failing is failing on master as well, so we'll ignore
> that. I will share the report shortly.
>
> They are the result of the job here, which I will summarize in a more
> readable format and mail out later tonight once it completes:
> http://jenkins.buildacloud.org/job/test-matrix/571/
>
> Each bubble in that grid represents a hypervisor profile against which
> all the tests were run.
>
> > Honestly this is all quite frustrating for myself being new to
> > contributing to ACS. I feel somewhat lost in the whole process of how
> > to get features in. I'll refrain from venting my frustrations.
> >
> > Darren
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Prasanna Santhanam <ts...@apache.org>.
On Tue, Oct 08, 2013 at 10:20:01AM -0700, Darren Shepherd wrote:
> >From what I can gather it seems that master currently fails the BVT
> (and know when I say BVT I mean that black box that apparently exists
> somewhere doing something, but I have no clue what it really means).
> So in turn my spring modularization branch will additionally fail BVT.
> Citrix internal QA ran some tests against my branch and they mostly
> passed but some failed. Its quite difficult to sort through this all
> because tests are failing on master. So I don't know what to do at
> this point. At least my branch won't completely blow up everything.
> I just know the longer it takes to merge this the more painful it will
> be....
Darren, Sorry about the frustrations. I haven't been able to keep
track these last few weeks about your work.
I have run the tests against your branch and everything looks good so
far. Whatever is failing is failing on master as well, so we'll ignore
that. I will share the report shortly.
They are the result of the job here, which I will summarize in a more
readable format and mail out later tonight once it completes:
http://jenkins.buildacloud.org/job/test-matrix/571/
Each bubble in that grid represents a hypervisor profile against which
all the tests were run.
> Honestly this is all quite frustrating for myself being new to
> contributing to ACS. I feel somewhat lost in the whole process of how
> to get features in. I'll refrain from venting my frustrations.
>
> Darren
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I'm personally +1. Simulator seems to work
On 10/8/13 3:12 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>So what's the verdict? What would it take for everyone to feel warm
>and fuzzy about this merge, given the troubled past this community has
>had with Spring. I'm not saying the code is perfect, but so far its
>not terribly bad :)
>
>Darren
>
>On Tue, Oct 8, 2013 at 11:10 AM, Chiradeep Vittal
><Ch...@citrix.com> wrote:
>> I'm not getting any notifications of BVT test failures. Where do I
>> subscribe?
>>
>> On 10/8/13 10:20 AM, "Darren Shepherd" <da...@gmail.com>
>>wrote:
>>
>> >From what I can gather it seems that master currently fails the BVT
>>>(and know when I say BVT I mean that black box that apparently exists
>>>somewhere doing something, but I have no clue what it really means).
>>>So in turn my spring modularization branch will additionally fail BVT.
>>> Citrix internal QA ran some tests against my branch and they mostly
>>>passed but some failed. Its quite difficult to sort through this all
>>>because tests are failing on master. So I don't know what to do at
>>>this point. At least my branch won't completely blow up everything.
>>>I just know the longer it takes to merge this the more painful it will
>>>be....
>>>
>>>Honestly this is all quite frustrating for myself being new to
>>>contributing to ACS. I feel somewhat lost in the whole process of how
>>>to get features in. I'll refrain from venting my frustrations.
>>>
>>>Darren
>>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
So what's the verdict? What would it take for everyone to feel warm
and fuzzy about this merge, given the troubled past this community has
had with Spring. I'm not saying the code is perfect, but so far its
not terribly bad :)
Darren
On Tue, Oct 8, 2013 at 11:10 AM, Chiradeep Vittal
<Ch...@citrix.com> wrote:
> I'm not getting any notifications of BVT test failures. Where do I
> subscribe?
>
> On 10/8/13 10:20 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>
> >From what I can gather it seems that master currently fails the BVT
>>(and know when I say BVT I mean that black box that apparently exists
>>somewhere doing something, but I have no clue what it really means).
>>So in turn my spring modularization branch will additionally fail BVT.
>> Citrix internal QA ran some tests against my branch and they mostly
>>passed but some failed. Its quite difficult to sort through this all
>>because tests are failing on master. So I don't know what to do at
>>this point. At least my branch won't completely blow up everything.
>>I just know the longer it takes to merge this the more painful it will
>>be....
>>
>>Honestly this is all quite frustrating for myself being new to
>>contributing to ACS. I feel somewhat lost in the whole process of how
>>to get features in. I'll refrain from venting my frustrations.
>>
>>Darren
>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I'm not getting any notifications of BVT test failures. Where do I
subscribe?
On 10/8/13 10:20 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>From what I can gather it seems that master currently fails the BVT
>(and know when I say BVT I mean that black box that apparently exists
>somewhere doing something, but I have no clue what it really means).
>So in turn my spring modularization branch will additionally fail BVT.
> Citrix internal QA ran some tests against my branch and they mostly
>passed but some failed. Its quite difficult to sort through this all
>because tests are failing on master. So I don't know what to do at
>this point. At least my branch won't completely blow up everything.
>I just know the longer it takes to merge this the more painful it will
>be....
>
>Honestly this is all quite frustrating for myself being new to
>contributing to ACS. I feel somewhat lost in the whole process of how
>to get features in. I'll refrain from venting my frustrations.
>
>Darren
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
>From what I can gather it seems that master currently fails the BVT
(and know when I say BVT I mean that black box that apparently exists
somewhere doing something, but I have no clue what it really means).
So in turn my spring modularization branch will additionally fail BVT.
Citrix internal QA ran some tests against my branch and they mostly
passed but some failed. Its quite difficult to sort through this all
because tests are failing on master. So I don't know what to do at
this point. At least my branch won't completely blow up everything.
I just know the longer it takes to merge this the more painful it will
be....
Honestly this is all quite frustrating for myself being new to
contributing to ACS. I feel somewhat lost in the whole process of how
to get features in. I'll refrain from venting my frustrations.
Darren
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
I just cherry picked that commit, so it should be fixed now.
Darren
On Fri, Oct 4, 2013 at 9:38 AM, Darren Shepherd
<da...@gmail.com> wrote:
> Yes unfortunately when I did the rebase and pushed the branch, master
> was broken. So that failing test is not related to Spring and was
> removed in commit 765f56a02ced44b7d95574f5901e834c799a56fe on master.
> Is there any way you can ignore the failure.
>
> Darren
>
> On Fri, Oct 4, 2013 at 4:13 AM, Prasanna Santhanam <ts...@apache.org> wrote:
>> Hey Darren, the build appears to be broken on spring-modularization.
>> Actually a test is broken. If fixed I could generate some packages for
>> testing.
>>
>> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-yumrepo-refresh/3028/console
>>
>> On Fri, Oct 04, 2013 at 10:27:55AM +0530, Prasanna Santhanam wrote:
>>> Thanks. I will first run a baseline against master since we don't have
>>> one and then one on spring modularization. The baseline against 4.2
>>> shows this result:
>>>
>>> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
>>>
>>> On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
>>> > This should be fixed on the spring-modularization branch.
>>> >
>>> > Darren
>>> >
>>> > On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam <ts...@apache.org> wrote:
>>> > > I switched the test infrastructure on jenkins.buildacloud.org to run
>>> > > the bvts [1] against master last week. Couple of weeks before that the
>>> > > simulator [2] tests were switched to run against master. Both are
>>> > > broken unfortunately and the bvt and checkin tests aren't running.
>>> > >
>>> > > I've filed a bug here:
>>> > > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
>>> > >
>>> > > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
>>> > > [2] http://jenkins.buildacloud.org/view/simulator/
>>> > >
>>> > > Anyone who has access to the jenkins can run the bvts on their desired
>>> > > branch. Simply login and change the test-yumrepo-refresh job to point
>>> > > to your branch. Build that to refresh the remote repository with the
>>> > > packages made from your branch. Then switch test-matrix to point to
>>> > > the same development branch and fire a build. That's about it.
>>> > >
>>> > > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
>>> > >> Yes agreed. I've extensively tested this, but that is never enough.
>>> > >> How do I get the BVTs ran against this. Due to the cross cutting
>>> > >> nature of this I want to get this merged as fast as possible.
>>> > >>
>>> > >> Darren
>>> > >>
>>> > >> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
>>> > >> >
>>> > >> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
>>> > >> >
>>> > >> > --Alex
>>> > >> >
>>> > >> >> -----Original Message-----
>>> > >> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
>>> > >> >> Sent: Wednesday, October 2, 2013 4:39 PM
>>> > >> >> To: dev@cloudstack.apache.org
>>> > >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
>>> > >> >> Modularization
>>> > >> >>
>>> > >> >> Darren,
>>> > >> >>
>>> > >> >> This looks really nice. A few questions on Spring AOP replacement.
>>> > >> >>
>>> > >> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>>> > >> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
>>> > >> >> places that are doing run-time type-casting, the code in these places
>>> > >> >> assumes a real object that implements all interfaces in the semantics context.
>>> > >> >> At the time when I initially converted to Spring, I couldn't ensure that
>>> > >> >> switching to proxy-based AOP can have 100% coverage for these run-time
>>> > >> >> cases. What is your approach to address this run-time type-casting issue?
>>> > >> >>
>>> > >> >> 2) We've run into a huge-memory footprint issue that may be caused by
>>> > >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
>>> > >> >> layer. Do you have a chance to run a memory analysis in the heap after
>>> > >> >> management server is started.
>>> > >> >>
>>> > >> >> I might be good to run BVT test on the branch before the merge, could
>>> > >> >> someone initiate the effort?
>>> > >> >>
>>> > >> >> kelven
>>> > >> >>
>>> > >> >>
>>> > >> >>
>>> > >> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
>>> > >> >> wrote:
>>> > >> >>
>>> > >> >>> Not sure how this works... I would like to merge in the new
>>> > >> >>> modularized Spring setup to master. There is info on the wiki about it
>>> > >> >>> [1] [2] [3]. The primary change is to break apart the monolithic
>>> > >> >>> applicationContext.xml and componentContext.xml files such that each
>>> > >> >>> plugin can maintain and contribute its own configuration.
>>> > >> >>>
>>> > >> >>> In addition to breaking up the configuration we no longer use ACS
>>> > >> >>> custom AOP and it is now fully Spring AOP.
>>> > >> >>>
>>> > >> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
>>> > >> >>> classpath (exception being commands.properties, I'll address that in a
>>> > >> >>> different thread). Unfortunately this branch does not have the changes
>>> > >> >>> to package things in different RPMs. So it would be great if somebody
>>> > >> >>> could take up the packaging effort to split out all the plugins into
>>> > >> >>> different RPMs.
>>> > >> >>>
>>> > >> >>> Darren
>>> > >> >>>
>>> > >> >>> [1]
>>> > >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
>>> > >> >> rin
>>> > >> >>> g
>>> > >> >>> [2]
>>> > >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
>>> > >> >> ins%2C+Modu
>>> > >> >>> les
>>> > >> >>> %2C+and+Extensions
>>> > >> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>>> > >> >
>>> > >
>>> > > --
>>> > > Prasanna.,
>>> > >
>>> > > ------------------------
>>> > > Powered by BigRock.com
>>> > >
>>>
>>> --
>>> Prasanna.,
>>>
>>> ------------------------
>>> Powered by BigRock.com
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
Yes unfortunately when I did the rebase and pushed the branch, master
was broken. So that failing test is not related to Spring and was
removed in commit 765f56a02ced44b7d95574f5901e834c799a56fe on master.
Is there any way you can ignore the failure.
Darren
On Fri, Oct 4, 2013 at 4:13 AM, Prasanna Santhanam <ts...@apache.org> wrote:
> Hey Darren, the build appears to be broken on spring-modularization.
> Actually a test is broken. If fixed I could generate some packages for
> testing.
>
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-yumrepo-refresh/3028/console
>
> On Fri, Oct 04, 2013 at 10:27:55AM +0530, Prasanna Santhanam wrote:
>> Thanks. I will first run a baseline against master since we don't have
>> one and then one on spring modularization. The baseline against 4.2
>> shows this result:
>>
>> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
>>
>> On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
>> > This should be fixed on the spring-modularization branch.
>> >
>> > Darren
>> >
>> > On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam <ts...@apache.org> wrote:
>> > > I switched the test infrastructure on jenkins.buildacloud.org to run
>> > > the bvts [1] against master last week. Couple of weeks before that the
>> > > simulator [2] tests were switched to run against master. Both are
>> > > broken unfortunately and the bvt and checkin tests aren't running.
>> > >
>> > > I've filed a bug here:
>> > > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
>> > >
>> > > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
>> > > [2] http://jenkins.buildacloud.org/view/simulator/
>> > >
>> > > Anyone who has access to the jenkins can run the bvts on their desired
>> > > branch. Simply login and change the test-yumrepo-refresh job to point
>> > > to your branch. Build that to refresh the remote repository with the
>> > > packages made from your branch. Then switch test-matrix to point to
>> > > the same development branch and fire a build. That's about it.
>> > >
>> > > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
>> > >> Yes agreed. I've extensively tested this, but that is never enough.
>> > >> How do I get the BVTs ran against this. Due to the cross cutting
>> > >> nature of this I want to get this merged as fast as possible.
>> > >>
>> > >> Darren
>> > >>
>> > >> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
>> > >> >
>> > >> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
>> > >> >
>> > >> > --Alex
>> > >> >
>> > >> >> -----Original Message-----
>> > >> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
>> > >> >> Sent: Wednesday, October 2, 2013 4:39 PM
>> > >> >> To: dev@cloudstack.apache.org
>> > >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
>> > >> >> Modularization
>> > >> >>
>> > >> >> Darren,
>> > >> >>
>> > >> >> This looks really nice. A few questions on Spring AOP replacement.
>> > >> >>
>> > >> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>> > >> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
>> > >> >> places that are doing run-time type-casting, the code in these places
>> > >> >> assumes a real object that implements all interfaces in the semantics context.
>> > >> >> At the time when I initially converted to Spring, I couldn't ensure that
>> > >> >> switching to proxy-based AOP can have 100% coverage for these run-time
>> > >> >> cases. What is your approach to address this run-time type-casting issue?
>> > >> >>
>> > >> >> 2) We've run into a huge-memory footprint issue that may be caused by
>> > >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
>> > >> >> layer. Do you have a chance to run a memory analysis in the heap after
>> > >> >> management server is started.
>> > >> >>
>> > >> >> I might be good to run BVT test on the branch before the merge, could
>> > >> >> someone initiate the effort?
>> > >> >>
>> > >> >> kelven
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
>> > >> >> wrote:
>> > >> >>
>> > >> >>> Not sure how this works... I would like to merge in the new
>> > >> >>> modularized Spring setup to master. There is info on the wiki about it
>> > >> >>> [1] [2] [3]. The primary change is to break apart the monolithic
>> > >> >>> applicationContext.xml and componentContext.xml files such that each
>> > >> >>> plugin can maintain and contribute its own configuration.
>> > >> >>>
>> > >> >>> In addition to breaking up the configuration we no longer use ACS
>> > >> >>> custom AOP and it is now fully Spring AOP.
>> > >> >>>
>> > >> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
>> > >> >>> classpath (exception being commands.properties, I'll address that in a
>> > >> >>> different thread). Unfortunately this branch does not have the changes
>> > >> >>> to package things in different RPMs. So it would be great if somebody
>> > >> >>> could take up the packaging effort to split out all the plugins into
>> > >> >>> different RPMs.
>> > >> >>>
>> > >> >>> Darren
>> > >> >>>
>> > >> >>> [1]
>> > >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
>> > >> >> rin
>> > >> >>> g
>> > >> >>> [2]
>> > >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
>> > >> >> ins%2C+Modu
>> > >> >>> les
>> > >> >>> %2C+and+Extensions
>> > >> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>> > >> >
>> > >
>> > > --
>> > > Prasanna.,
>> > >
>> > > ------------------------
>> > > Powered by BigRock.com
>> > >
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Prasanna Santhanam <ts...@apache.org>.
Hey Darren, the build appears to be broken on spring-modularization.
Actually a test is broken. If fixed I could generate some packages for
testing.
http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-yumrepo-refresh/3028/console
On Fri, Oct 04, 2013 at 10:27:55AM +0530, Prasanna Santhanam wrote:
> Thanks. I will first run a baseline against master since we don't have
> one and then one on spring modularization. The baseline against 4.2
> shows this result:
>
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
>
> On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
> > This should be fixed on the spring-modularization branch.
> >
> > Darren
> >
> > On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam <ts...@apache.org> wrote:
> > > I switched the test infrastructure on jenkins.buildacloud.org to run
> > > the bvts [1] against master last week. Couple of weeks before that the
> > > simulator [2] tests were switched to run against master. Both are
> > > broken unfortunately and the bvt and checkin tests aren't running.
> > >
> > > I've filed a bug here:
> > > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
> > >
> > > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> > > [2] http://jenkins.buildacloud.org/view/simulator/
> > >
> > > Anyone who has access to the jenkins can run the bvts on their desired
> > > branch. Simply login and change the test-yumrepo-refresh job to point
> > > to your branch. Build that to refresh the remote repository with the
> > > packages made from your branch. Then switch test-matrix to point to
> > > the same development branch and fire a build. That's about it.
> > >
> > > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> > >> Yes agreed. I've extensively tested this, but that is never enough.
> > >> How do I get the BVTs ran against this. Due to the cross cutting
> > >> nature of this I want to get this merged as fast as possible.
> > >>
> > >> Darren
> > >>
> > >> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
> > >> >
> > >> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
> > >> >
> > >> > --Alex
> > >> >
> > >> >> -----Original Message-----
> > >> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
> > >> >> Sent: Wednesday, October 2, 2013 4:39 PM
> > >> >> To: dev@cloudstack.apache.org
> > >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> > >> >> Modularization
> > >> >>
> > >> >> Darren,
> > >> >>
> > >> >> This looks really nice. A few questions on Spring AOP replacement.
> > >> >>
> > >> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> > >> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
> > >> >> places that are doing run-time type-casting, the code in these places
> > >> >> assumes a real object that implements all interfaces in the semantics context.
> > >> >> At the time when I initially converted to Spring, I couldn't ensure that
> > >> >> switching to proxy-based AOP can have 100% coverage for these run-time
> > >> >> cases. What is your approach to address this run-time type-casting issue?
> > >> >>
> > >> >> 2) We've run into a huge-memory footprint issue that may be caused by
> > >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
> > >> >> layer. Do you have a chance to run a memory analysis in the heap after
> > >> >> management server is started.
> > >> >>
> > >> >> I might be good to run BVT test on the branch before the merge, could
> > >> >> someone initiate the effort?
> > >> >>
> > >> >> kelven
> > >> >>
> > >> >>
> > >> >>
> > >> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
> > >> >> wrote:
> > >> >>
> > >> >>> Not sure how this works... I would like to merge in the new
> > >> >>> modularized Spring setup to master. There is info on the wiki about it
> > >> >>> [1] [2] [3]. The primary change is to break apart the monolithic
> > >> >>> applicationContext.xml and componentContext.xml files such that each
> > >> >>> plugin can maintain and contribute its own configuration.
> > >> >>>
> > >> >>> In addition to breaking up the configuration we no longer use ACS
> > >> >>> custom AOP and it is now fully Spring AOP.
> > >> >>>
> > >> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
> > >> >>> classpath (exception being commands.properties, I'll address that in a
> > >> >>> different thread). Unfortunately this branch does not have the changes
> > >> >>> to package things in different RPMs. So it would be great if somebody
> > >> >>> could take up the packaging effort to split out all the plugins into
> > >> >>> different RPMs.
> > >> >>>
> > >> >>> Darren
> > >> >>>
> > >> >>> [1]
> > >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> > >> >> rin
> > >> >>> g
> > >> >>> [2]
> > >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> > >> >> ins%2C+Modu
> > >> >>> les
> > >> >>> %2C+and+Extensions
> > >> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> > >> >
> > >
> > > --
> > > Prasanna.,
> > >
> > > ------------------------
> > > Powered by BigRock.com
> > >
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Prasanna Santhanam <ts...@apache.org>.
Thanks. I will first run a baseline against master since we don't have
one and then one on spring modularization. The baseline against 4.2
shows this result:
http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/252/testReport/
On Thu, Oct 03, 2013 at 09:19:23AM -0700, Darren Shepherd wrote:
> This should be fixed on the spring-modularization branch.
>
> Darren
>
> On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam <ts...@apache.org> wrote:
> > I switched the test infrastructure on jenkins.buildacloud.org to run
> > the bvts [1] against master last week. Couple of weeks before that the
> > simulator [2] tests were switched to run against master. Both are
> > broken unfortunately and the bvt and checkin tests aren't running.
> >
> > I've filed a bug here:
> > https://issues.apache.org/jira//browse/CLOUDSTACK-4791
> >
> > [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> > [2] http://jenkins.buildacloud.org/view/simulator/
> >
> > Anyone who has access to the jenkins can run the bvts on their desired
> > branch. Simply login and change the test-yumrepo-refresh job to point
> > to your branch. Build that to refresh the remote repository with the
> > packages made from your branch. Then switch test-matrix to point to
> > the same development branch and fire a build. That's about it.
> >
> > On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> >> Yes agreed. I've extensively tested this, but that is never enough.
> >> How do I get the BVTs ran against this. Due to the cross cutting
> >> nature of this I want to get this merged as fast as possible.
> >>
> >> Darren
> >>
> >> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
> >> >
> >> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
> >> >
> >> > --Alex
> >> >
> >> >> -----Original Message-----
> >> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
> >> >> Sent: Wednesday, October 2, 2013 4:39 PM
> >> >> To: dev@cloudstack.apache.org
> >> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> >> >> Modularization
> >> >>
> >> >> Darren,
> >> >>
> >> >> This looks really nice. A few questions on Spring AOP replacement.
> >> >>
> >> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> >> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
> >> >> places that are doing run-time type-casting, the code in these places
> >> >> assumes a real object that implements all interfaces in the semantics context.
> >> >> At the time when I initially converted to Spring, I couldn't ensure that
> >> >> switching to proxy-based AOP can have 100% coverage for these run-time
> >> >> cases. What is your approach to address this run-time type-casting issue?
> >> >>
> >> >> 2) We've run into a huge-memory footprint issue that may be caused by
> >> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
> >> >> layer. Do you have a chance to run a memory analysis in the heap after
> >> >> management server is started.
> >> >>
> >> >> I might be good to run BVT test on the branch before the merge, could
> >> >> someone initiate the effort?
> >> >>
> >> >> kelven
> >> >>
> >> >>
> >> >>
> >> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Not sure how this works... I would like to merge in the new
> >> >>> modularized Spring setup to master. There is info on the wiki about it
> >> >>> [1] [2] [3]. The primary change is to break apart the monolithic
> >> >>> applicationContext.xml and componentContext.xml files such that each
> >> >>> plugin can maintain and contribute its own configuration.
> >> >>>
> >> >>> In addition to breaking up the configuration we no longer use ACS
> >> >>> custom AOP and it is now fully Spring AOP.
> >> >>>
> >> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
> >> >>> classpath (exception being commands.properties, I'll address that in a
> >> >>> different thread). Unfortunately this branch does not have the changes
> >> >>> to package things in different RPMs. So it would be great if somebody
> >> >>> could take up the packaging effort to split out all the plugins into
> >> >>> different RPMs.
> >> >>>
> >> >>> Darren
> >> >>>
> >> >>> [1]
> >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> >> >> rin
> >> >>> g
> >> >>> [2]
> >> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >> >> ins%2C+Modu
> >> >>> les
> >> >>> %2C+and+Extensions
> >> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> >> >
> >
> > --
> > Prasanna.,
> >
> > ------------------------
> > Powered by BigRock.com
> >
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
This should be fixed on the spring-modularization branch.
Darren
On Wed, Oct 2, 2013 at 10:05 PM, Prasanna Santhanam <ts...@apache.org> wrote:
> I switched the test infrastructure on jenkins.buildacloud.org to run
> the bvts [1] against master last week. Couple of weeks before that the
> simulator [2] tests were switched to run against master. Both are
> broken unfortunately and the bvt and checkin tests aren't running.
>
> I've filed a bug here:
> https://issues.apache.org/jira//browse/CLOUDSTACK-4791
>
> [1] http://jenkins.buildacloud.org/view/cloudstack-qa/
> [2] http://jenkins.buildacloud.org/view/simulator/
>
> Anyone who has access to the jenkins can run the bvts on their desired
> branch. Simply login and change the test-yumrepo-refresh job to point
> to your branch. Build that to refresh the remote repository with the
> packages made from your branch. Then switch test-matrix to point to
> the same development branch and fire a build. That's about it.
>
> On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
>> Yes agreed. I've extensively tested this, but that is never enough.
>> How do I get the BVTs ran against this. Due to the cross cutting
>> nature of this I want to get this merged as fast as possible.
>>
>> Darren
>>
>> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
>> >
>> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
>> >
>> > --Alex
>> >
>> >> -----Original Message-----
>> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
>> >> Sent: Wednesday, October 2, 2013 4:39 PM
>> >> To: dev@cloudstack.apache.org
>> >> Subject: Re: [MERGE] spring-modularization to master - Spring
>> >> Modularization
>> >>
>> >> Darren,
>> >>
>> >> This looks really nice. A few questions on Spring AOP replacement.
>> >>
>> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
>> >> places that are doing run-time type-casting, the code in these places
>> >> assumes a real object that implements all interfaces in the semantics context.
>> >> At the time when I initially converted to Spring, I couldn't ensure that
>> >> switching to proxy-based AOP can have 100% coverage for these run-time
>> >> cases. What is your approach to address this run-time type-casting issue?
>> >>
>> >> 2) We've run into a huge-memory footprint issue that may be caused by
>> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
>> >> layer. Do you have a chance to run a memory analysis in the heap after
>> >> management server is started.
>> >>
>> >> I might be good to run BVT test on the branch before the merge, could
>> >> someone initiate the effort?
>> >>
>> >> kelven
>> >>
>> >>
>> >>
>> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
>> >> wrote:
>> >>
>> >>> Not sure how this works... I would like to merge in the new
>> >>> modularized Spring setup to master. There is info on the wiki about it
>> >>> [1] [2] [3]. The primary change is to break apart the monolithic
>> >>> applicationContext.xml and componentContext.xml files such that each
>> >>> plugin can maintain and contribute its own configuration.
>> >>>
>> >>> In addition to breaking up the configuration we no longer use ACS
>> >>> custom AOP and it is now fully Spring AOP.
>> >>>
>> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
>> >>> classpath (exception being commands.properties, I'll address that in a
>> >>> different thread). Unfortunately this branch does not have the changes
>> >>> to package things in different RPMs. So it would be great if somebody
>> >>> could take up the packaging effort to split out all the plugins into
>> >>> different RPMs.
>> >>>
>> >>> Darren
>> >>>
>> >>> [1]
>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
>> >> rin
>> >>> g
>> >>> [2]
>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
>> >> ins%2C+Modu
>> >>> les
>> >>> %2C+and+Extensions
>> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>> >
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Prasanna Santhanam <ts...@apache.org>.
I switched the test infrastructure on jenkins.buildacloud.org to run
the bvts [1] against master last week. Couple of weeks before that the
simulator [2] tests were switched to run against master. Both are
broken unfortunately and the bvt and checkin tests aren't running.
I've filed a bug here:
https://issues.apache.org/jira//browse/CLOUDSTACK-4791
[1] http://jenkins.buildacloud.org/view/cloudstack-qa/
[2] http://jenkins.buildacloud.org/view/simulator/
Anyone who has access to the jenkins can run the bvts on their desired
branch. Simply login and change the test-yumrepo-refresh job to point
to your branch. Build that to refresh the remote repository with the
packages made from your branch. Then switch test-matrix to point to
the same development branch and fire a build. That's about it.
On Wed, Oct 02, 2013 at 05:42:54PM -0700, Darren Shepherd wrote:
> Yes agreed. I've extensively tested this, but that is never enough.
> How do I get the BVTs ran against this. Due to the cross cutting
> nature of this I want to get this merged as fast as possible.
>
> Darren
>
> > On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
> >
> > +1 on running the BVT on it. We've been through this one once before. Should be careful.
> >
> > --Alex
> >
> >> -----Original Message-----
> >> From: Kelven Yang [mailto:kelven.yang@citrix.com]
> >> Sent: Wednesday, October 2, 2013 4:39 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [MERGE] spring-modularization to master - Spring
> >> Modularization
> >>
> >> Darren,
> >>
> >> This looks really nice. A few questions on Spring AOP replacement.
> >>
> >> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> >> AOP is mainly due to that inside existing CloudStack codebase, we have many
> >> places that are doing run-time type-casting, the code in these places
> >> assumes a real object that implements all interfaces in the semantics context.
> >> At the time when I initially converted to Spring, I couldn't ensure that
> >> switching to proxy-based AOP can have 100% coverage for these run-time
> >> cases. What is your approach to address this run-time type-casting issue?
> >>
> >> 2) We've run into a huge-memory footprint issue that may be caused by
> >> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
> >> layer. Do you have a chance to run a memory analysis in the heap after
> >> management server is started.
> >>
> >> I might be good to run BVT test on the branch before the merge, could
> >> someone initiate the effort?
> >>
> >> kelven
> >>
> >>
> >>
> >> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
> >> wrote:
> >>
> >>> Not sure how this works... I would like to merge in the new
> >>> modularized Spring setup to master. There is info on the wiki about it
> >>> [1] [2] [3]. The primary change is to break apart the monolithic
> >>> applicationContext.xml and componentContext.xml files such that each
> >>> plugin can maintain and contribute its own configuration.
> >>>
> >>> In addition to breaking up the configuration we no longer use ACS
> >>> custom AOP and it is now fully Spring AOP.
> >>>
> >>> Now adding/removing a plug-in is a matter of just adding a jar to the
> >>> classpath (exception being commands.properties, I'll address that in a
> >>> different thread). Unfortunately this branch does not have the changes
> >>> to package things in different RPMs. So it would be great if somebody
> >>> could take up the packaging effort to split out all the plugins into
> >>> different RPMs.
> >>>
> >>> Darren
> >>>
> >>> [1]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> >> rin
> >>> g
> >>> [2]
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >> ins%2C+Modu
> >>> les
> >>> %2C+and+Extensions
> >>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
> >
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
Yes agreed. I've extensively tested this, but that is never enough. How do I get the BVTs ran against this. Due to the cross cutting nature of this I want to get this merged as fast as possible.
Darren
> On Oct 2, 2013, at 4:43 PM, Alex Huang <Al...@citrix.com> wrote:
>
> +1 on running the BVT on it. We've been through this one once before. Should be careful.
>
> --Alex
>
>> -----Original Message-----
>> From: Kelven Yang [mailto:kelven.yang@citrix.com]
>> Sent: Wednesday, October 2, 2013 4:39 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [MERGE] spring-modularization to master - Spring
>> Modularization
>>
>> Darren,
>>
>> This looks really nice. A few questions on Spring AOP replacement.
>>
>> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>> AOP is mainly due to that inside existing CloudStack codebase, we have many
>> places that are doing run-time type-casting, the code in these places
>> assumes a real object that implements all interfaces in the semantics context.
>> At the time when I initially converted to Spring, I couldn't ensure that
>> switching to proxy-based AOP can have 100% coverage for these run-time
>> cases. What is your approach to address this run-time type-casting issue?
>>
>> 2) We've run into a huge-memory footprint issue that may be caused by
>> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
>> layer. Do you have a chance to run a memory analysis in the heap after
>> management server is started.
>>
>> I might be good to run BVT test on the branch before the merge, could
>> someone initiate the effort?
>>
>> kelven
>>
>>
>>
>> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
>> wrote:
>>
>>> Not sure how this works... I would like to merge in the new
>>> modularized Spring setup to master. There is info on the wiki about it
>>> [1] [2] [3]. The primary change is to break apart the monolithic
>>> applicationContext.xml and componentContext.xml files such that each
>>> plugin can maintain and contribute its own configuration.
>>>
>>> In addition to breaking up the configuration we no longer use ACS
>>> custom AOP and it is now fully Spring AOP.
>>>
>>> Now adding/removing a plug-in is a matter of just adding a jar to the
>>> classpath (exception being commands.properties, I'll address that in a
>>> different thread). Unfortunately this branch does not have the changes
>>> to package things in different RPMs. So it would be great if somebody
>>> could take up the packaging effort to split out all the plugins into
>>> different RPMs.
>>>
>>> Darren
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
>> rin
>>> g
>>> [2]
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
>> ins%2C+Modu
>>> les
>>> %2C+and+Extensions
>>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>
RE: [MERGE] spring-modularization to master - Spring Modularization
Posted by Alex Huang <Al...@citrix.com>.
+1 on running the BVT on it. We've been through this one once before. Should be careful.
--Alex
> -----Original Message-----
> From: Kelven Yang [mailto:kelven.yang@citrix.com]
> Sent: Wednesday, October 2, 2013 4:39 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [MERGE] spring-modularization to master - Spring
> Modularization
>
> Darren,
>
> This looks really nice. A few questions on Spring AOP replacement.
>
> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> AOP is mainly due to that inside existing CloudStack codebase, we have many
> places that are doing run-time type-casting, the code in these places
> assumes a real object that implements all interfaces in the semantics context.
> At the time when I initially converted to Spring, I couldn't ensure that
> switching to proxy-based AOP can have 100% coverage for these run-time
> cases. What is your approach to address this run-time type-casting issue?
>
> 2) We've run into a huge-memory footprint issue that may be caused by
> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack Dao
> layer. Do you have a chance to run a memory analysis in the heap after
> management server is started.
>
> I might be good to run BVT test on the branch before the merge, could
> someone initiate the effort?
>
> kelven
>
>
>
> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
> wrote:
>
> >Not sure how this works... I would like to merge in the new
> >modularized Spring setup to master. There is info on the wiki about it
> >[1] [2] [3]. The primary change is to break apart the monolithic
> >applicationContext.xml and componentContext.xml files such that each
> >plugin can maintain and contribute its own configuration.
> >
> >In addition to breaking up the configuration we no longer use ACS
> >custom AOP and it is now fully Spring AOP.
> >
> >Now adding/removing a plug-in is a matter of just adding a jar to the
> >classpath (exception being commands.properties, I'll address that in a
> >different thread). Unfortunately this branch does not have the changes
> >to package things in different RPMs. So it would be great if somebody
> >could take up the packaging effort to split out all the plugins into
> >different RPMs.
> >
> >Darren
> >
> >[1]
> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sp
> rin
> >g
> >[2]
> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> ins%2C+Modu
> >les
> >%2C+and+Extensions
> >[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Kelven Yang <ke...@citrix.com>.
On 10/2/13 5:52 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>Kelvin,
>
>Also since I'm using dynamic proxies for AOP, spring is suppose to expose
>all interface of the target. But I'll still dig in and make sure 100%
>this is all good.
>
>Darren
>
>> On Oct 2, 2013, at 5:40 PM, Darren Shepherd
>><da...@gmail.com> wrote:
>>
>> I know the majority of the polymorphic stuff is working fine right now.
>> But now that you bring it up I'm kinda wondering why.
I brought it up just for cautious reason, since I know we have quite many
places that do run-time type check and casting, it relies heavily on the
semantic that the target object should be 100% compatible to real object
if there is proxy business involved.
One of the other reason is that since these are run-time checks, testing
without enough coverage may not reveal all the problems.
-Kelven
>>It very we'll be possible that the way in which I'm discovering
>>extensible types precludes it from AOP matching the beans. (Which
>>isn't bad. We should only use AOP for declarative transaction
>>management ideally. AOP on a project this size is most always a bad
>>idea). The AOP is far more selective now anyhow. Only beans of
>>GenericDaoBase and those that have @ActionEvent are being matched which
>>isn't really any of the extensible types that require instanceof checks.
>> Regardless I'll dig into exactly what is happening and give you an
>>answer to be sure that issue won't hit us.
>>
>> I've done extensive profiling on this. The heap and permgen are
>>slightly less than what they used to be. I've been running -Xmx256mb
>>for all my testing. Heap analysis shows that with all plugins
>>(including noredist), permgen is at about 120mb and heap usage is about
>>150mb.
>>
>> Regarding the start up time, it is slightly slower. It's about 5
>>seconds slower to start on my laptop than before. I traced down the
>>problem to AOP. Because I'm selectively looking for @ActionEvent it has
>>to evaluate every method on every bean, which is slow. I know how to
>>optimize it such that the startup can actually be faster than what it
>>was before, but I really want to focus on correctness over speed right
>>now.
>>
>> Cglib is no longer used for AOP. The only use of cglib now is in
>>GenericDaoBase when it creates the VOs and also in the async framework.
>>
>> I should point out all of these changes apply only to the mgmt server.
>>Awsapi and usage server are untouched. I did not have time to convert
>>them too. So those still use ACS AOP and the monolithic config (which
>>uses component scanning, so nobody remove @Component unless your really
>>sure it's not used).
>>
>> Darren
>>
>>> On Oct 2, 2013, at 4:39 PM, Kelven Yang <ke...@citrix.com> wrote:
>>>
>>> Darren,
>>>
>>> This looks really nice. A few questions on Spring AOP replacement.
>>>
>>> 1) Spring AOP is proxy-based, the reason we ended up of using
>>>customized
>>> AOP is mainly due to that inside existing CloudStack codebase, we have
>>> many places that are doing run-time type-casting, the code in these
>>>places
>>> assumes a real object that implements all interfaces in the semantics
>>> context. At the time when I initially converted to Spring, I couldn't
>>> ensure that switching to proxy-based AOP can have 100% coverage for
>>>these
>>> run-time cases. What is your approach to address this run-time
>>> type-casting issue?
>>>
>>> 2) We've run into a huge-memory footprint issue that may be caused by
>>> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in
>>>CloudStack
>>> Dao layer. Do you have a chance to run a memory analysis in the heap
>>>after
>>> management server is started.
>>>
>>> I might be good to run BVT test on the branch before the merge, could
>>> someone initiate the effort?
>>>
>>> kelven
>>>
>>>
>>>
>>>> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com>
>>>>wrote:
>>>>
>>>> Not sure how this works... I would like to merge in the new
>>>> modularized Spring setup to master. There is info on the wiki about it
>>>> [1] [2] [3]. The primary change is to break apart the monolithic
>>>> applicationContext.xml and componentContext.xml files such that each
>>>> plugin can maintain and contribute its own configuration.
>>>>
>>>> In addition to breaking up the configuration we no longer use ACS
>>>> custom AOP and it is now fully Spring AOP.
>>>>
>>>> Now adding/removing a plug-in is a matter of just adding a jar to the
>>>> classpath (exception being commands.properties, I'll address that in a
>>>> different thread). Unfortunately this branch does not have the
>>>> changes to package things in different RPMs. So it would be great if
>>>> somebody could take up the packaging effort to split out all the
>>>> plugins into different RPMs.
>>>>
>>>> Darren
>>>>
>>>> [1]
>>>>
>>>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Sprin
>>>>g
>>>> [2]
>>>>
>>>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-ins%2C+Modu
>>>>les
>>>> %2C+and+Extensions
>>>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>>>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
Kelvin,
Also since I'm using dynamic proxies for AOP, spring is suppose to expose all interface of the target. But I'll still dig in and make sure 100% this is all good.
Darren
> On Oct 2, 2013, at 5:40 PM, Darren Shepherd <da...@gmail.com> wrote:
>
> I know the majority of the polymorphic stuff is working fine right now. But now that you bring it up I'm kinda wondering why. It very we'll be possible that the way in which I'm discovering extensible types precludes it from AOP matching the beans. (Which isn't bad. We should only use AOP for declarative transaction management ideally. AOP on a project this size is most always a bad idea). The AOP is far more selective now anyhow. Only beans of GenericDaoBase and those that have @ActionEvent are being matched which isn't really any of the extensible types that require instanceof checks. Regardless I'll dig into exactly what is happening and give you an answer to be sure that issue won't hit us.
>
> I've done extensive profiling on this. The heap and permgen are slightly less than what they used to be. I've been running -Xmx256mb for all my testing. Heap analysis shows that with all plugins (including noredist), permgen is at about 120mb and heap usage is about 150mb.
>
> Regarding the start up time, it is slightly slower. It's about 5 seconds slower to start on my laptop than before. I traced down the problem to AOP. Because I'm selectively looking for @ActionEvent it has to evaluate every method on every bean, which is slow. I know how to optimize it such that the startup can actually be faster than what it was before, but I really want to focus on correctness over speed right now.
>
> Cglib is no longer used for AOP. The only use of cglib now is in GenericDaoBase when it creates the VOs and also in the async framework.
>
> I should point out all of these changes apply only to the mgmt server. Awsapi and usage server are untouched. I did not have time to convert them too. So those still use ACS AOP and the monolithic config (which uses component scanning, so nobody remove @Component unless your really sure it's not used).
>
> Darren
>
>> On Oct 2, 2013, at 4:39 PM, Kelven Yang <ke...@citrix.com> wrote:
>>
>> Darren,
>>
>> This looks really nice. A few questions on Spring AOP replacement.
>>
>> 1) Spring AOP is proxy-based, the reason we ended up of using customized
>> AOP is mainly due to that inside existing CloudStack codebase, we have
>> many places that are doing run-time type-casting, the code in these places
>> assumes a real object that implements all interfaces in the semantics
>> context. At the time when I initially converted to Spring, I couldn't
>> ensure that switching to proxy-based AOP can have 100% coverage for these
>> run-time cases. What is your approach to address this run-time
>> type-casting issue?
>>
>> 2) We've run into a huge-memory footprint issue that may be caused by
>> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack
>> Dao layer. Do you have a chance to run a memory analysis in the heap after
>> management server is started.
>>
>> I might be good to run BVT test on the branch before the merge, could
>> someone initiate the effort?
>>
>> kelven
>>
>>
>>
>>> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>>>
>>> Not sure how this works... I would like to merge in the new
>>> modularized Spring setup to master. There is info on the wiki about it
>>> [1] [2] [3]. The primary change is to break apart the monolithic
>>> applicationContext.xml and componentContext.xml files such that each
>>> plugin can maintain and contribute its own configuration.
>>>
>>> In addition to breaking up the configuration we no longer use ACS
>>> custom AOP and it is now fully Spring AOP.
>>>
>>> Now adding/removing a plug-in is a matter of just adding a jar to the
>>> classpath (exception being commands.properties, I'll address that in a
>>> different thread). Unfortunately this branch does not have the
>>> changes to package things in different RPMs. So it would be great if
>>> somebody could take up the packaging effort to split out all the
>>> plugins into different RPMs.
>>>
>>> Darren
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring
>>> [2]
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-ins%2C+Modules
>>> %2C+and+Extensions
>>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Darren Shepherd <da...@gmail.com>.
I know the majority of the polymorphic stuff is working fine right now. But now that you bring it up I'm kinda wondering why. It very we'll be possible that the way in which I'm discovering extensible types precludes it from AOP matching the beans. (Which isn't bad. We should only use AOP for declarative transaction management ideally. AOP on a project this size is most always a bad idea). The AOP is far more selective now anyhow. Only beans of GenericDaoBase and those that have @ActionEvent are being matched which isn't really any of the extensible types that require instanceof checks. Regardless I'll dig into exactly what is happening and give you an answer to be sure that issue won't hit us.
I've done extensive profiling on this. The heap and permgen are slightly less than what they used to be. I've been running -Xmx256mb for all my testing. Heap analysis shows that with all plugins (including noredist), permgen is at about 120mb and heap usage is about 150mb.
Regarding the start up time, it is slightly slower. It's about 5 seconds slower to start on my laptop than before. I traced down the problem to AOP. Because I'm selectively looking for @ActionEvent it has to evaluate every method on every bean, which is slow. I know how to optimize it such that the startup can actually be faster than what it was before, but I really want to focus on correctness over speed right now.
Cglib is no longer used for AOP. The only use of cglib now is in GenericDaoBase when it creates the VOs and also in the async framework.
I should point out all of these changes apply only to the mgmt server. Awsapi and usage server are untouched. I did not have time to convert them too. So those still use ACS AOP and the monolithic config (which uses component scanning, so nobody remove @Component unless your really sure it's not used).
Darren
> On Oct 2, 2013, at 4:39 PM, Kelven Yang <ke...@citrix.com> wrote:
>
> Darren,
>
> This looks really nice. A few questions on Spring AOP replacement.
>
> 1) Spring AOP is proxy-based, the reason we ended up of using customized
> AOP is mainly due to that inside existing CloudStack codebase, we have
> many places that are doing run-time type-casting, the code in these places
> assumes a real object that implements all interfaces in the semantics
> context. At the time when I initially converted to Spring, I couldn't
> ensure that switching to proxy-based AOP can have 100% coverage for these
> run-time cases. What is your approach to address this run-time
> type-casting issue?
>
> 2) We've run into a huge-memory footprint issue that may be caused by
> conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack
> Dao layer. Do you have a chance to run a memory analysis in the heap after
> management server is started.
>
> I might be good to run BVT test on the branch before the merge, could
> someone initiate the effort?
>
> kelven
>
>
>
>> On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>>
>> Not sure how this works... I would like to merge in the new
>> modularized Spring setup to master. There is info on the wiki about it
>> [1] [2] [3]. The primary change is to break apart the monolithic
>> applicationContext.xml and componentContext.xml files such that each
>> plugin can maintain and contribute its own configuration.
>>
>> In addition to breaking up the configuration we no longer use ACS
>> custom AOP and it is now fully Spring AOP.
>>
>> Now adding/removing a plug-in is a matter of just adding a jar to the
>> classpath (exception being commands.properties, I'll address that in a
>> different thread). Unfortunately this branch does not have the
>> changes to package things in different RPMs. So it would be great if
>> somebody could take up the packaging effort to split out all the
>> plugins into different RPMs.
>>
>> Darren
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring
>> [2]
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-ins%2C+Modules
>> %2C+and+Extensions
>> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions
>
Re: [MERGE] spring-modularization to master - Spring Modularization
Posted by Kelven Yang <ke...@citrix.com>.
Darren,
This looks really nice. A few questions on Spring AOP replacement.
1) Spring AOP is proxy-based, the reason we ended up of using customized
AOP is mainly due to that inside existing CloudStack codebase, we have
many places that are doing run-time type-casting, the code in these places
assumes a real object that implements all interfaces in the semantics
context. At the time when I initially converted to Spring, I couldn't
ensure that switching to proxy-based AOP can have 100% coverage for these
run-time cases. What is your approach to address this run-time
type-casting issue?
2) We've run into a huge-memory footprint issue that may be caused by
conflicts of CGLIB usage in Spring AOP and the CGLIB usage in CloudStack
Dao layer. Do you have a chance to run a memory analysis in the heap after
management server is started.
I might be good to run BVT test on the branch before the merge, could
someone initiate the effort?
kelven
On 10/2/13 3:48 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>Not sure how this works... I would like to merge in the new
>modularized Spring setup to master. There is info on the wiki about it
>[1] [2] [3]. The primary change is to break apart the monolithic
>applicationContext.xml and componentContext.xml files such that each
>plugin can maintain and contribute its own configuration.
>
>In addition to breaking up the configuration we no longer use ACS
>custom AOP and it is now fully Spring AOP.
>
>Now adding/removing a plug-in is a matter of just adding a jar to the
>classpath (exception being commands.properties, I'll address that in a
>different thread). Unfortunately this branch does not have the
>changes to package things in different RPMs. So it would be great if
>somebody could take up the packaging effort to split out all the
>plugins into different RPMs.
>
>Darren
>
>[1]
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring
>[2]
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-ins%2C+Modules
>%2C+and+Extensions
>[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extensions