You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by John Burwell <jo...@shapeblue.com> on 2016/06/17 18:56:16 UTC

4.9/master Testing Coordination

All,

It is a bit lo-fi, but if you are testing master in preparation for the 4.9 RC, could you please share information about the configurations you testing (e.g. hypervisors, storage backends, network configurations, etc)?  Any test results could also be helpful.  The hope is to reduce duplication of effort and understand how much of the system has been covered.

Thanks,
-John
john.burwell@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue



Re: 4.9/master Testing Coordination

Posted by Ron Wheeler <rw...@artifact-software.com>.
Would it make sense to use the wiki for tracking this?

Ron

On 17/06/2016 2:56 PM, John Burwell wrote:
> All,
>
> It is a bit lo-fi, but if you are testing master in preparation for the 4.9 RC, could you please share information about the configurations you testing (e.g. hypervisors, storage backends, network configurations, etc)?  Any test results could also be helpful.  The hope is to reduce duplication of effort and understand how much of the system has been covered.
>
> Thanks,
> -John
> john.burwell@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>


-- 
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102


Re: 4.9/master Testing Coordination

Posted by Syed Mushtaq <sy...@gmail.com>.
I am testing the following configuration:

CentOS 6.7 Mgmt Server + XenServer 6.5 + SolidFire (using Managed pluign) +
Swift (Secondary Storage).

I've hit the following issue when uploading templates:

https://github.com/apache/cloudstack/pull/1598/

Would someone be able to review this please?

Thanks,
-Syed


On Thu, Jun 23, 2016 at 4:01 PM, John Burwell <jo...@shapeblue.com>
wrote:

> All,
>
> I had hoped to get testing off the ground earlier, but $dayjob duties have
> gotten in the way.  Over the weekend, I am planning to kick off tests of
> the following configurations:
>
>         • CentOS 6.8 management server + CentOS 6.8 KVM Hosts using NFS
> primary and secondary storage (would allow us to verify/fix the documented
> libvirt/qemu versions)
>         • CentOS 6.8 management server + vCenter 5.5u3d + ESXi 5.5u3b
> using NFS primary and secondary storage
>         • CentOS 6.8 management server + vCenter 6.0u2 + ESXi Express
> Patch 6 using NFS primary and secondary storage
>         • CentOS 6.8 management server + XenServer 6.2 SP1 using NFS
> primary and secondary storage
>         • CentOS 6.8 management server + XenServer 6.5 SP1 using NFS
> primary and secondary storage
>
> In each of these environments, I plan to run the following tests:
>
>         • All smoke tests
>         • Component Tests
>                 • test_accounts.py
>                 • test_acl_*.py
>                 • test_sharednetwork*.py
>                 • test_add_remove_network.py
>                 • test_advancedsg_networks.py
>                 • test_affinity_groups*.py
>                 • test_cpu_domain_limits.py
>                 • test_cpu_limits.py
>                 • test_cpu_max_limits.py
>                 • test_host_maintenance.py
>                 • test_memory_limits.py
>                 • test_network_offering.py
>                 • test_overcommit.py
>                 • test_persistent_networks.py
>                 • test_ps_domain_limits.py
>                 • test_ps_limits.py
>                 • test_ps_max_limits.py
>                 • test_ps_resize_volume.py
>                 • test_ps_resource_limits_volume.py
>                 • test_resource_limits.py
>                 • test_routers.py
>                 • test_security_groups.py
>                 • test_shared_networks.py
>                 • test_snapshots.py
>                 • test_ss_domain_limits.py
>                 • test_ss_limits.py
>                 • test_ss_max_limits.py
>                 • test_templates.py
>                 • test_update_vm.py
>                 • test_volumes.py
>                 • test_vpc.py
>
> I will start this set, and may adjust based on success rates and runtime.
>
> I hope to post results by COB, Monday (27 June 2016).
>
> Thanks,
> -John
>
>
> >
> john.burwell@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>
> On Jun 20, 2016, at 4:31 PM, John Burwell <jo...@shapeblue.com>
> wrote:
> >
> > All,
> >
> > I am working to coordinate some testing at ShapeBlue.  I send an update
> as soon as I have the list of environment and tests we plan to run.
> >
> > Thanks,
> > -John
> >
> >>
> > john.burwell@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > On Jun 20, 2016, at 8:27 AM, Simon Weller <sw...@ena.com> wrote:
> >>
> >>
> >> Remi,
> >>
> >>
> >> The KVM VXLAN feature uses the standard BridgeVifDriver, as OVS doesn't
> support multicast.
> >>
> >>
> >> To reproduce this:
> >>
> >>
> >> Deploy 4.9 RPMS to Centos 7.2 installation, upgrading a 4.8 install.
> >>
> >> Restart existing VPC router pair for a established VPC.
> >>
> >> When VPC routers come back up, the guest tier (isolated) network is
> missing.
> >>
> >>
> >> If you deploy a new VPC, the same problem occurs.
> >>
> >>
> >> We have confirmed that the same problem exists on a non-redundant VPC
> router as well.
> >>
> >>
> >> David is working on reproducing this in a bubble. We have reproduced
> this on 2 hardware labs thus far.
> >>
> >>
> >> - Si
> >>
> >> ________________________________
> >> From: Remi Bergsma <RB...@schubergphilis.com>
> >> Sent: Saturday, June 18, 2016 2:58 AM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: 4.9/master Testing Coordination
> >>
> >> Hi Simon,
> >>
> >> Do you have the exact stept to reproduce?
> >> To me it sounds like the issue is in either the ovsVifDriver or the
> VXLAN stuff. Can you reproduce the scenario in the bubble (with its default
> vlan/bridgeVifDriver)?
> >>
> >> If there is a clear scenario, I think we should write an integration
> test (even if that shows it’s broken).
> >>
> >> Regards, Remi
> >>
> >>
> >> On 17/06/16 22:47, "Simon Weller" <sw...@ena.com> wrote:
> >>
> >>> Here's a quick run down on the configuration(s) we're testing:
> >>>
> >>>
> >>> Centos 7.2
> >>>
> >>> Advanced Zone with VXLAN on KVM
> >>>
> >>> VPC functionality including Private GW, VPN, Static Routes, ACL Lists
> et al
> >>>
> >>> Redundant VPC VRs
> >>>
> >>> Ceph Primary Storage
> >>>
> >>> NFS and S3 secondary storage
> >>>
> >>> As Will mentioned, we've found an odd issue with VPCs that we're still
> debugging.
> >>>
> >>> Here's a summary of what we've found thus far:
> >>> When a tier is added, the network interface for the tier network is
> never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2
> is never plugged when you attempt to provision the first VM within the VPC
> and the VM creation fails. If you have existing VMs and you restart the
> router, you lose the eth2 interface in the libvirt configuration (confirmed
> with a virsh dumpxml).
> >>> If you leave the VRs alone after the upgrade, VMs can be provisioned
> correctly (until you reboot the VRs).
> >>>
> >>> I did also run into the NIO SSL agent not connecting problem again.
> When I telnetted to 8250, the agent immediately came up without me having
> to restart it. So keep an eye out for that as well.
> >>>
> >>>
> >>> - Si
> >>>
> >>> ________________________________
> >>> From: williamstevens@gmail.com <wi...@gmail.com> on behalf
> of Will Stevens <ws...@cloudops.com>
> >>> Sent: Friday, June 17, 2016 3:02 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: Re: 4.9/master Testing Coordination
> >>>
> >>> Syed is looking into a potential issue with Swift as secondary storage
> on
> >>> master.
> >>>
> >>> ENA is looking into a potential problem when upgrading a VR which was
> >>> working in 4.8.0, but after an upgrade to 4.9.0 and restarting the
> network
> >>> there are only 2 nics instead of 3.  If they spin a new VR from
> scratch it
> >>> seems to work.  I need to follow up with them to see if they have an
> >>> updated status of their testing.
> >>>
> >>> *Will STEVENS*
> >>> Lead Developer
> >>>
> >>> *CloudOps* *| *Cloud Solutions Experts
> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>> w cloudops.com *|* tw @CloudOps_
> >>>
> >>> On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> wrote:
> >>>
> >>>> The following tests are running cleanly on KVM with Advanced
> networking
> >>>> and 2 hosts.
> >>>>
> >>>> echo "Running tests with required_hardware=true"
> >>>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> >>>> tags=advanced,required_hardware=true \
> >>>> smoke/test_password_server.py \
> >>>> smoke/test_vpc_redundant.py \
> >>>> smoke/test_routers_iptables_default_policy.py \
> >>>> smoke/test_routers_network_ops.py \
> >>>> smoke/test_vpc_router_nics.py \
> >>>> smoke/test_router_dhcphosts.py \
> >>>> smoke/test_loadbalance.py \
> >>>> smoke/test_internal_lb.py \
> >>>> smoke/test_ssvm.py \
> >>>> smoke/test_vpc_vpn.py \
> >>>> smoke/test_privategw_acl.py \
> >>>> smoke/test_network.py
> >>>>
> >>>> echo "Running tests with required_hardware=false"
> >>>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> >>>> tags=advanced,required_hardware=false \
> >>>> smoke/test_routers.py \
> >>>> smoke/test_network_acl.py \
> >>>> smoke/test_reset_vm_on_reboot.py \
> >>>> smoke/test_vm_life_cycle.py \
> >>>> smoke/test_service_offerings.py \
> >>>> smoke/test_network.py \
> >>>> component/test_vpc_offerings.py \
> >>>> component/test_vpc_routers.py
> >>>>
> >>>> I need to do some more manual testing...
> >>>>
> >>>> *Will STEVENS*
> >>>> Lead Developer
> >>>>
> >>>> *CloudOps* *| *Cloud Solutions Experts
> >>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>>> w cloudops.com *|* tw @CloudOps_
> >>>>
> >>>> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> >>>> Mike.Tutkowski@netapp.com> wrote:
> >>>>
> >>>>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> >>>>>
> >>>>> I executed all of the tests in test/integration/plugins/solidfire.
> >>>>>
> >>>>> They all came back successful.
> >>>>> ________________________________________
> >>>>> From: John Burwell <jo...@shapeblue.com>
> >>>>> Sent: Friday, June 17, 2016 12:56 PM
> >>>>> To: dev@cloudstack.apache.org
> >>>>> Subject: 4.9/master Testing Coordination
> >>>>>
> >>>>> All,
> >>>>>
> >>>>> It is a bit lo-fi, but if you are testing master in preparation for
> the
> >>>>> 4.9 RC, could you please share information about the configurations
> you
> >>>>> testing (e.g. hypervisors, storage backends, network configurations,
> etc)?
> >>>>> Any test results could also be helpful.  The hope is to reduce
> duplication
> >>>>> of effort and understand how much of the system has been covered.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>> john.burwell@shapeblue.com
> >>>>> www.shapeblue.com<http://www.shapeblue.com>
> >> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> >> www.shapeblue.com
> >> Overview Apache CloudStack contains an authentication module providing
> “single sign-on” functionality via the SAML data format. Under certain
> conditions, a
> >>
> >>
> >>
> >>> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> >>> www.shapeblue.com<http://www.shapeblue.com>
> >>> Overview Apache CloudStack contains an authentication module providing
> "single sign-on" functionality via the SAML data format. Under certain
> conditions, a
> >>>
> >>>
> >>>
> >>>>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> >>>>> @shapeblue
> >
> >
>
>
>

Re: 4.9/master Testing Coordination

Posted by John Burwell <jo...@shapeblue.com>.
All,

I had hoped to get testing off the ground earlier, but $dayjob duties have gotten in the way.  Over the weekend, I am planning to kick off tests of the following configurations:

	• CentOS 6.8 management server + CentOS 6.8 KVM Hosts using NFS primary and secondary storage (would allow us to verify/fix the documented libvirt/qemu versions)
	• CentOS 6.8 management server + vCenter 5.5u3d + ESXi 5.5u3b using NFS primary and secondary storage
	• CentOS 6.8 management server + vCenter 6.0u2 + ESXi Express Patch 6 using NFS primary and secondary storage
	• CentOS 6.8 management server + XenServer 6.2 SP1 using NFS primary and secondary storage
	• CentOS 6.8 management server + XenServer 6.5 SP1 using NFS primary and secondary storage

In each of these environments, I plan to run the following tests:

	• All smoke tests
	• Component Tests
		• test_accounts.py
		• test_acl_*.py
		• test_sharednetwork*.py
		• test_add_remove_network.py
		• test_advancedsg_networks.py
		• test_affinity_groups*.py
		• test_cpu_domain_limits.py
		• test_cpu_limits.py
		• test_cpu_max_limits.py
		• test_host_maintenance.py
		• test_memory_limits.py
		• test_network_offering.py
		• test_overcommit.py
		• test_persistent_networks.py
		• test_ps_domain_limits.py
		• test_ps_limits.py
		• test_ps_max_limits.py
		• test_ps_resize_volume.py
		• test_ps_resource_limits_volume.py
		• test_resource_limits.py
		• test_routers.py
		• test_security_groups.py
		• test_shared_networks.py
		• test_snapshots.py
		• test_ss_domain_limits.py
		• test_ss_limits.py
		• test_ss_max_limits.py
		• test_templates.py
		• test_update_vm.py
		• test_volumes.py
		• test_vpc.py

I will start this set, and may adjust based on success rates and runtime.

I hope to post results by COB, Monday (27 June 2016).

Thanks,
-John


> 
john.burwell@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
  
 

On Jun 20, 2016, at 4:31 PM, John Burwell <jo...@shapeblue.com> wrote:
> 
> All,
> 
> I am working to coordinate some testing at ShapeBlue.  I send an update as soon as I have the list of environment and tests we plan to run.
> 
> Thanks,
> -John
> 
>> 
> john.burwell@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
> 
> 
> 
> On Jun 20, 2016, at 8:27 AM, Simon Weller <sw...@ena.com> wrote:
>> 
>> 
>> Remi,
>> 
>> 
>> The KVM VXLAN feature uses the standard BridgeVifDriver, as OVS doesn't support multicast.
>> 
>> 
>> To reproduce this:
>> 
>> 
>> Deploy 4.9 RPMS to Centos 7.2 installation, upgrading a 4.8 install.
>> 
>> Restart existing VPC router pair for a established VPC.
>> 
>> When VPC routers come back up, the guest tier (isolated) network is missing.
>> 
>> 
>> If you deploy a new VPC, the same problem occurs.
>> 
>> 
>> We have confirmed that the same problem exists on a non-redundant VPC router as well.
>> 
>> 
>> David is working on reproducing this in a bubble. We have reproduced this on 2 hardware labs thus far.
>> 
>> 
>> - Si
>> 
>> ________________________________
>> From: Remi Bergsma <RB...@schubergphilis.com>
>> Sent: Saturday, June 18, 2016 2:58 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: 4.9/master Testing Coordination
>> 
>> Hi Simon,
>> 
>> Do you have the exact stept to reproduce?
>> To me it sounds like the issue is in either the ovsVifDriver or the VXLAN stuff. Can you reproduce the scenario in the bubble (with its default vlan/bridgeVifDriver)?
>> 
>> If there is a clear scenario, I think we should write an integration test (even if that shows it’s broken).
>> 
>> Regards, Remi
>> 
>> 
>> On 17/06/16 22:47, "Simon Weller" <sw...@ena.com> wrote:
>> 
>>> Here's a quick run down on the configuration(s) we're testing:
>>> 
>>> 
>>> Centos 7.2
>>> 
>>> Advanced Zone with VXLAN on KVM
>>> 
>>> VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al
>>> 
>>> Redundant VPC VRs
>>> 
>>> Ceph Primary Storage
>>> 
>>> NFS and S3 secondary storage
>>> 
>>> As Will mentioned, we've found an odd issue with VPCs that we're still debugging.
>>> 
>>> Here's a summary of what we've found thus far:
>>> When a tier is added, the network interface for the tier network is never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is never plugged when you attempt to provision the first VM within the VPC and the VM creation fails. If you have existing VMs and you restart the router, you lose the eth2 interface in the libvirt configuration (confirmed with a virsh dumpxml).
>>> If you leave the VRs alone after the upgrade, VMs can be provisioned correctly (until you reboot the VRs).
>>> 
>>> I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.
>>> 
>>> 
>>> - Si
>>> 
>>> ________________________________
>>> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
>>> Sent: Friday, June 17, 2016 3:02 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: 4.9/master Testing Coordination
>>> 
>>> Syed is looking into a potential issue with Swift as secondary storage on
>>> master.
>>> 
>>> ENA is looking into a potential problem when upgrading a VR which was
>>> working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
>>> there are only 2 nics instead of 3.  If they spin a new VR from scratch it
>>> seems to work.  I need to follow up with them to see if they have an
>>> updated status of their testing.
>>> 
>>> *Will STEVENS*
>>> Lead Developer
>>> 
>>> *CloudOps* *| *Cloud Solutions Experts
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>> w cloudops.com *|* tw @CloudOps_
>>> 
>>> On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:
>>> 
>>>> The following tests are running cleanly on KVM with Advanced networking
>>>> and 2 hosts.
>>>> 
>>>> echo "Running tests with required_hardware=true"
>>>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>>>> tags=advanced,required_hardware=true \
>>>> smoke/test_password_server.py \
>>>> smoke/test_vpc_redundant.py \
>>>> smoke/test_routers_iptables_default_policy.py \
>>>> smoke/test_routers_network_ops.py \
>>>> smoke/test_vpc_router_nics.py \
>>>> smoke/test_router_dhcphosts.py \
>>>> smoke/test_loadbalance.py \
>>>> smoke/test_internal_lb.py \
>>>> smoke/test_ssvm.py \
>>>> smoke/test_vpc_vpn.py \
>>>> smoke/test_privategw_acl.py \
>>>> smoke/test_network.py
>>>> 
>>>> echo "Running tests with required_hardware=false"
>>>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>>>> tags=advanced,required_hardware=false \
>>>> smoke/test_routers.py \
>>>> smoke/test_network_acl.py \
>>>> smoke/test_reset_vm_on_reboot.py \
>>>> smoke/test_vm_life_cycle.py \
>>>> smoke/test_service_offerings.py \
>>>> smoke/test_network.py \
>>>> component/test_vpc_offerings.py \
>>>> component/test_vpc_routers.py
>>>> 
>>>> I need to do some more manual testing...
>>>> 
>>>> *Will STEVENS*
>>>> Lead Developer
>>>> 
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>> w cloudops.com *|* tw @CloudOps_
>>>> 
>>>> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
>>>> Mike.Tutkowski@netapp.com> wrote:
>>>> 
>>>>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>>>> 
>>>>> I executed all of the tests in test/integration/plugins/solidfire.
>>>>> 
>>>>> They all came back successful.
>>>>> ________________________________________
>>>>> From: John Burwell <jo...@shapeblue.com>
>>>>> Sent: Friday, June 17, 2016 12:56 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: 4.9/master Testing Coordination
>>>>> 
>>>>> All,
>>>>> 
>>>>> It is a bit lo-fi, but if you are testing master in preparation for the
>>>>> 4.9 RC, could you please share information about the configurations you
>>>>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>>>>> Any test results could also be helpful.  The hope is to reduce duplication
>>>>> of effort and understand how much of the system has been covered.
>>>>> 
>>>>> Thanks,
>>>>> -John
>>>>> john.burwell@shapeblue.com
>>>>> www.shapeblue.com<http://www.shapeblue.com>
>> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
>> www.shapeblue.com
>> Overview Apache CloudStack contains an authentication module providing “single sign-on” functionality via the SAML data format. Under certain conditions, a
>> 
>> 
>> 
>>> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
>>> www.shapeblue.com<http://www.shapeblue.com>
>>> Overview Apache CloudStack contains an authentication module providing "single sign-on" functionality via the SAML data format. Under certain conditions, a
>>> 
>>> 
>>> 
>>>>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>>>>> @shapeblue
> 
> 



Re: 4.9/master Testing Coordination

Posted by John Burwell <jo...@shapeblue.com>.
All,

I am working to coordinate some testing at ShapeBlue.  I send an update as soon as I have the list of environment and tests we plan to run.

Thanks,
-John

> 
john.burwell@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
  
 

On Jun 20, 2016, at 8:27 AM, Simon Weller <sw...@ena.com> wrote:
> 
> 
> Remi,
> 
> 
> The KVM VXLAN feature uses the standard BridgeVifDriver, as OVS doesn't support multicast.
> 
> 
> To reproduce this:
> 
> 
> Deploy 4.9 RPMS to Centos 7.2 installation, upgrading a 4.8 install.
> 
> Restart existing VPC router pair for a established VPC.
> 
> When VPC routers come back up, the guest tier (isolated) network is missing.
> 
> 
> If you deploy a new VPC, the same problem occurs.
> 
> 
> We have confirmed that the same problem exists on a non-redundant VPC router as well.
> 
> 
> David is working on reproducing this in a bubble. We have reproduced this on 2 hardware labs thus far.
> 
> 
> - Si
> 
> ________________________________
> From: Remi Bergsma <RB...@schubergphilis.com>
> Sent: Saturday, June 18, 2016 2:58 AM
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
> 
> Hi Simon,
> 
> Do you have the exact stept to reproduce?
> To me it sounds like the issue is in either the ovsVifDriver or the VXLAN stuff. Can you reproduce the scenario in the bubble (with its default vlan/bridgeVifDriver)?
> 
> If there is a clear scenario, I think we should write an integration test (even if that shows it’s broken).
> 
> Regards, Remi
> 
> 
> On 17/06/16 22:47, "Simon Weller" <sw...@ena.com> wrote:
> 
>> Here's a quick run down on the configuration(s) we're testing:
>> 
>> 
>> Centos 7.2
>> 
>> Advanced Zone with VXLAN on KVM
>> 
>> VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al
>> 
>> Redundant VPC VRs
>> 
>> Ceph Primary Storage
>> 
>> NFS and S3 secondary storage
>> 
>> As Will mentioned, we've found an odd issue with VPCs that we're still debugging.
>> 
>> Here's a summary of what we've found thus far:
>> When a tier is added, the network interface for the tier network is never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is never plugged when you attempt to provision the first VM within the VPC and the VM creation fails. If you have existing VMs and you restart the router, you lose the eth2 interface in the libvirt configuration (confirmed with a virsh dumpxml).
>> If you leave the VRs alone after the upgrade, VMs can be provisioned correctly (until you reboot the VRs).
>> 
>> I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.
>> 
>> 
>> - Si
>> 
>> ________________________________
>> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
>> Sent: Friday, June 17, 2016 3:02 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: 4.9/master Testing Coordination
>> 
>> Syed is looking into a potential issue with Swift as secondary storage on
>> master.
>> 
>> ENA is looking into a potential problem when upgrading a VR which was
>> working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
>> there are only 2 nics instead of 3.  If they spin a new VR from scratch it
>> seems to work.  I need to follow up with them to see if they have an
>> updated status of their testing.
>> 
>> *Will STEVENS*
>> Lead Developer
>> 
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>> 
>> On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:
>> 
>>> The following tests are running cleanly on KVM with Advanced networking
>>> and 2 hosts.
>>> 
>>> echo "Running tests with required_hardware=true"
>>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>>> tags=advanced,required_hardware=true \
>>> smoke/test_password_server.py \
>>> smoke/test_vpc_redundant.py \
>>> smoke/test_routers_iptables_default_policy.py \
>>> smoke/test_routers_network_ops.py \
>>> smoke/test_vpc_router_nics.py \
>>> smoke/test_router_dhcphosts.py \
>>> smoke/test_loadbalance.py \
>>> smoke/test_internal_lb.py \
>>> smoke/test_ssvm.py \
>>> smoke/test_vpc_vpn.py \
>>> smoke/test_privategw_acl.py \
>>> smoke/test_network.py
>>> 
>>> echo "Running tests with required_hardware=false"
>>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>>> tags=advanced,required_hardware=false \
>>> smoke/test_routers.py \
>>> smoke/test_network_acl.py \
>>> smoke/test_reset_vm_on_reboot.py \
>>> smoke/test_vm_life_cycle.py \
>>> smoke/test_service_offerings.py \
>>> smoke/test_network.py \
>>> component/test_vpc_offerings.py \
>>> component/test_vpc_routers.py
>>> 
>>> I need to do some more manual testing...
>>> 
>>> *Will STEVENS*
>>> Lead Developer
>>> 
>>> *CloudOps* *| *Cloud Solutions Experts
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>> w cloudops.com *|* tw @CloudOps_
>>> 
>>> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
>>> Mike.Tutkowski@netapp.com> wrote:
>>> 
>>>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>>> 
>>>> I executed all of the tests in test/integration/plugins/solidfire.
>>>> 
>>>> They all came back successful.
>>>> ________________________________________
>>>> From: John Burwell <jo...@shapeblue.com>
>>>> Sent: Friday, June 17, 2016 12:56 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: 4.9/master Testing Coordination
>>>> 
>>>> All,
>>>> 
>>>> It is a bit lo-fi, but if you are testing master in preparation for the
>>>> 4.9 RC, could you please share information about the configurations you
>>>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>>>> Any test results could also be helpful.  The hope is to reduce duplication
>>>> of effort and understand how much of the system has been covered.
>>>> 
>>>> Thanks,
>>>> -John
>>>> john.burwell@shapeblue.com
>>>> www.shapeblue.com<http://www.shapeblue.com>
> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> www.shapeblue.com
> Overview Apache CloudStack contains an authentication module providing “single sign-on” functionality via the SAML data format. Under certain conditions, a
> 
> 
> 
>> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
>> www.shapeblue.com<http://www.shapeblue.com>
>> Overview Apache CloudStack contains an authentication module providing "single sign-on" functionality via the SAML data format. Under certain conditions, a
>> 
>> 
>> 
>>>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>>>> @shapeblue



Re: 4.9/master Testing Coordination

Posted by Simon Weller <sw...@ena.com>.
Remi,


The KVM VXLAN feature uses the standard BridgeVifDriver, as OVS doesn't support multicast.


To reproduce this:


Deploy 4.9 RPMS to Centos 7.2 installation, upgrading a 4.8 install.

Restart existing VPC router pair for a established VPC.

When VPC routers come back up, the guest tier (isolated) network is missing.


If you deploy a new VPC, the same problem occurs.


We have confirmed that the same problem exists on a non-redundant VPC router as well.


David is working on reproducing this in a bubble. We have reproduced this on 2 hardware labs thus far.


- Si

________________________________
From: Remi Bergsma <RB...@schubergphilis.com>
Sent: Saturday, June 18, 2016 2:58 AM
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

Hi Simon,

Do you have the exact stept to reproduce?
To me it sounds like the issue is in either the ovsVifDriver or the VXLAN stuff. Can you reproduce the scenario in the bubble (with its default vlan/bridgeVifDriver)?

If there is a clear scenario, I think we should write an integration test (even if that shows it’s broken).

Regards, Remi


On 17/06/16 22:47, "Simon Weller" <sw...@ena.com> wrote:

>Here's a quick run down on the configuration(s) we're testing:
>
>
>Centos 7.2
>
>Advanced Zone with VXLAN on KVM
>
>VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al
>
>Redundant VPC VRs
>
>Ceph Primary Storage
>
>NFS and S3 secondary storage
>
>As Will mentioned, we've found an odd issue with VPCs that we're still debugging.
>
>Here's a summary of what we've found thus far:
>When a tier is added, the network interface for the tier network is never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is never plugged when you attempt to provision the first VM within the VPC and the VM creation fails. If you have existing VMs and you restart the router, you lose the eth2 interface in the libvirt configuration (confirmed with a virsh dumpxml).
>If you leave the VRs alone after the upgrade, VMs can be provisioned correctly (until you reboot the VRs).
>
>I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.
>
>
>- Si
>
>________________________________
>From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
>Sent: Friday, June 17, 2016 3:02 PM
>To: dev@cloudstack.apache.org
>Subject: Re: 4.9/master Testing Coordination
>
>Syed is looking into a potential issue with Swift as secondary storage on
>master.
>
>ENA is looking into a potential problem when upgrading a VR which was
>working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
>there are only 2 nics instead of 3.  If they spin a new VR from scratch it
>seems to work.  I need to follow up with them to see if they have an
>updated status of their testing.
>
>*Will STEVENS*
>Lead Developer
>
>*CloudOps* *| *Cloud Solutions Experts
>420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>w cloudops.com *|* tw @CloudOps_
>
>On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:
>
>> The following tests are running cleanly on KVM with Advanced networking
>> and 2 hosts.
>>
>> echo "Running tests with required_hardware=true"
>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>> tags=advanced,required_hardware=true \
>> smoke/test_password_server.py \
>> smoke/test_vpc_redundant.py \
>> smoke/test_routers_iptables_default_policy.py \
>> smoke/test_routers_network_ops.py \
>> smoke/test_vpc_router_nics.py \
>> smoke/test_router_dhcphosts.py \
>> smoke/test_loadbalance.py \
>> smoke/test_internal_lb.py \
>> smoke/test_ssvm.py \
>> smoke/test_vpc_vpn.py \
>> smoke/test_privategw_acl.py \
>> smoke/test_network.py
>>
>> echo "Running tests with required_hardware=false"
>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>> tags=advanced,required_hardware=false \
>> smoke/test_routers.py \
>> smoke/test_network_acl.py \
>> smoke/test_reset_vm_on_reboot.py \
>> smoke/test_vm_life_cycle.py \
>> smoke/test_service_offerings.py \
>> smoke/test_network.py \
>> component/test_vpc_offerings.py \
>> component/test_vpc_routers.py
>>
>> I need to do some more manual testing...
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
>> Mike.Tutkowski@netapp.com> wrote:
>>
>>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>>
>>> I executed all of the tests in test/integration/plugins/solidfire.
>>>
>>> They all came back successful.
>>> ________________________________________
>>> From: John Burwell <jo...@shapeblue.com>
>>> Sent: Friday, June 17, 2016 12:56 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: 4.9/master Testing Coordination
>>>
>>> All,
>>>
>>> It is a bit lo-fi, but if you are testing master in preparation for the
>>> 4.9 RC, could you please share information about the configurations you
>>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>>> Any test results could also be helpful.  The hope is to reduce duplication
>>> of effort and understand how much of the system has been covered.
>>>
>>> Thanks,
>>> -John
>>> john.burwell@shapeblue.com
>>> www.shapeblue.com<http://www.shapeblue.com>
ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
www.shapeblue.com
Overview Apache CloudStack contains an authentication module providing “single sign-on” functionality via the SAML data format. Under certain conditions, a



>ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
>www.shapeblue.com<http://www.shapeblue.com>
>Overview Apache CloudStack contains an authentication module providing "single sign-on" functionality via the SAML data format. Under certain conditions, a
>
>
>
>>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>>
>>


Re: 4.9/master Testing Coordination

Posted by Remi Bergsma <RB...@schubergphilis.com>.
Hi Simon,

Do you have the exact stept to reproduce?
To me it sounds like the issue is in either the ovsVifDriver or the VXLAN stuff. Can you reproduce the scenario in the bubble (with its default vlan/bridgeVifDriver)?

If there is a clear scenario, I think we should write an integration test (even if that shows it’s broken).

Regards, Remi


On 17/06/16 22:47, "Simon Weller" <sw...@ena.com> wrote:

>Here's a quick run down on the configuration(s) we're testing:
>
>
>Centos 7.2
>
>Advanced Zone with VXLAN on KVM
>
>VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al
>
>Redundant VPC VRs
>
>Ceph Primary Storage
>
>NFS and S3 secondary storage
>
>As Will mentioned, we've found an odd issue with VPCs that we're still debugging.
>
>Here's a summary of what we've found thus far:
>When a tier is added, the network interface for the tier network is never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is never plugged when you attempt to provision the first VM within the VPC and the VM creation fails. If you have existing VMs and you restart the router, you lose the eth2 interface in the libvirt configuration (confirmed with a virsh dumpxml).
>If you leave the VRs alone after the upgrade, VMs can be provisioned correctly (until you reboot the VRs).
>
>I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.
>
>
>- Si
>
>________________________________
>From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
>Sent: Friday, June 17, 2016 3:02 PM
>To: dev@cloudstack.apache.org
>Subject: Re: 4.9/master Testing Coordination
>
>Syed is looking into a potential issue with Swift as secondary storage on
>master.
>
>ENA is looking into a potential problem when upgrading a VR which was
>working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
>there are only 2 nics instead of 3.  If they spin a new VR from scratch it
>seems to work.  I need to follow up with them to see if they have an
>updated status of their testing.
>
>*Will STEVENS*
>Lead Developer
>
>*CloudOps* *| *Cloud Solutions Experts
>420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>w cloudops.com *|* tw @CloudOps_
>
>On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:
>
>> The following tests are running cleanly on KVM with Advanced networking
>> and 2 hosts.
>>
>> echo "Running tests with required_hardware=true"
>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>> tags=advanced,required_hardware=true \
>> smoke/test_password_server.py \
>> smoke/test_vpc_redundant.py \
>> smoke/test_routers_iptables_default_policy.py \
>> smoke/test_routers_network_ops.py \
>> smoke/test_vpc_router_nics.py \
>> smoke/test_router_dhcphosts.py \
>> smoke/test_loadbalance.py \
>> smoke/test_internal_lb.py \
>> smoke/test_ssvm.py \
>> smoke/test_vpc_vpn.py \
>> smoke/test_privategw_acl.py \
>> smoke/test_network.py
>>
>> echo "Running tests with required_hardware=false"
>> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
>> tags=advanced,required_hardware=false \
>> smoke/test_routers.py \
>> smoke/test_network_acl.py \
>> smoke/test_reset_vm_on_reboot.py \
>> smoke/test_vm_life_cycle.py \
>> smoke/test_service_offerings.py \
>> smoke/test_network.py \
>> component/test_vpc_offerings.py \
>> component/test_vpc_routers.py
>>
>> I need to do some more manual testing...
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
>> Mike.Tutkowski@netapp.com> wrote:
>>
>>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>>
>>> I executed all of the tests in test/integration/plugins/solidfire.
>>>
>>> They all came back successful.
>>> ________________________________________
>>> From: John Burwell <jo...@shapeblue.com>
>>> Sent: Friday, June 17, 2016 12:56 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: 4.9/master Testing Coordination
>>>
>>> All,
>>>
>>> It is a bit lo-fi, but if you are testing master in preparation for the
>>> 4.9 RC, could you please share information about the configurations you
>>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>>> Any test results could also be helpful.  The hope is to reduce duplication
>>> of effort and understand how much of the system has been covered.
>>>
>>> Thanks,
>>> -John
>>> john.burwell@shapeblue.com
>>> www.shapeblue.com<http://www.shapeblue.com>
>ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
>www.shapeblue.com
>Overview Apache CloudStack contains an authentication module providing "single sign-on" functionality via the SAML data format. Under certain conditions, a
>
>
>
>>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>>
>>


RE: 4.9/master Testing Coordination

Posted by Paul Angus <pa...@shapeblue.com>.
Np. Happy 'tenting'  :)


Kind regards,

Paul Angus

paul.angus@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: 29 June 2016 15:16
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

Great, thanks for the support Paul.  :)

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_

On Wed, Jun 29, 2016 at 10:12 AM, Paul Angus <pa...@shapeblue.com>
wrote:

> Thanks for the update Will.
> I'll look for the VR issue, I think that I've it in 4.5 intermittently 
> on vmware. - in fact I've seen it come up with 6 interfaces before...
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: 29 June 2016 15:09
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> We are keeping an eye on the NIO SSH stuff because we have had reports of
> the connection being lost for more than 10 minutes (before manual
> intervention), but it seems intermittent.
>
> There is also a potential issue with the VR coming up with only two nics
> instead of three, but there are conflicting reports on that one, so keep an
> eye out for that too.
>
> Most of the details of the things people have come up against are in this
> thread.
>
> I will be camping till monday, so I will have very limited connectivity.
> I will hopefully cut the RC when I get back, so please let me know if you
> find anything this week.
>
> Thanks,
>
> Will
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Wed, Jun 29, 2016 at 9:15 AM, Paul Angus <pa...@shapeblue.com>
> wrote:
>
> > Hi Will,
> >
> > What's the current situation with the 4.9 release?
> > I'm helping John with setting up the environments which he's listed
> > for testing. Do we have a list of any blocker bugs that need clearing
> > or a timetable?
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -----Original Message-----
> > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: 28 June 2016 18:16
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Rohit,
> > Simon and his team is in the final push getting their next internal
> > production release ready to ship, so their testing of 4.9 has paused
> while
> > they get that ready.  I suspect he and his team will have limited time to
> > dedicate to this in this week.
> >
> > Just figured I would pass that along if he is not able to get back to you
> > quickly so you understand why there may be a delay.  :)
> >
> > Cheers,
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
> > wrote:
> >
> > > Hi Simon,
> > >
> > >
> > > Thanks for sharing. I'm continuing the discussion from a PR (
> > > https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> > > here wrt NIO ssl agent connection. Are you using latest master, and
> > > are you able to consistently reproduce the issue? Make sure that both
> > > management server and kvm-agent (or ssvm agents etc.) are using the
> > > packages built from latest master as well (i.e. updating both agent
> > > and mgmt server pkg jars is necessary).
> > >
> > >
> > > > ""I did also run into the NIO SSL agent not connecting problem again.
> > > When I telnetted to 8250, the agent immediately came up without me
> > > having to restart it. So keep an eye out for that as well.""
> > >
> > >
> > > When you ran into the issue, what was the environment like. Did you
> > > have to wait long enough before you decided to telnet on agent's port
> > > 8250? The current master's NioConnection code makes sure to check for
> > > new channels/connections every one second, this change ensures that
> > > reconnections are fast and non-blocking.
> > >
> > >
> > > Regards.
> > >
> > > ________________________________
> > > From: Simon Weller <sw...@ena.com>
> > > Sent: 18 June 2016 02:17:08
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: 4.9/master Testing Coordination
> > >
> > > Here's a quick run down on the configuration(s) we're testing:
> > >
> > >
> > > Centos 7.2
> > >
> > > Advanced Zone with VXLAN on KVM
> > >
> > > VPC functionality including Private GW, VPN, Static Routes, ACL Lists
> > > et al
> > >
> > > Redundant VPC VRs
> > >
> > > Ceph Primary Storage
> > >
> > > NFS and S3 secondary storage
> > >
> > > As Will mentioned, we've found an odd issue with VPCs that we're still
> > > debugging.
> > >
> > > Here's a summary of what we've found thus far:
> > > When a tier is added, the network interface for the tier network is
> > > never plugged by libvirt. You only get 2 interfaces (eth0 and eth1).
> > > eth2 is never plugged when you attempt to provision the first VM
> > > within the VPC and the VM creation fails. If you have existing VMs and
> > > you restart the router, you lose the eth2 interface in the libvirt
> > > configuration (confirmed with a virsh dumpxml).
> > > If you leave the VRs alone after the upgrade, VMs can be provisioned
> > > correctly (until you reboot the VRs).
> > >
> > > I did also run into the NIO SSL agent not connecting problem again.
> > > When I telnetted to 8250, the agent immediately came up without me
> > > having to restart it. So keep an eye out for that as well.
> > >
> > >
> > > - Si
> > >
> > > ________________________________
> > > From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> > > Will Stevens <ws...@cloudops.com>
> > > Sent: Friday, June 17, 2016 3:02 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: 4.9/master Testing Coordination
> > >
> > > Syed is looking into a potential issue with Swift as secondary storage
> > > on master.
> > >
> > > ENA is looking into a potential problem when upgrading a VR which was
> > > working in 4.8.0, but after an upgrade to 4.9.0 and restarting the
> > > network there are only 2 nics instead of 3.  If they spin a new VR
> > > from scratch it seems to work.  I need to follow up with them to see
> > > if they have an updated status of their testing.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > > @CloudOps_
> > >
> > >
> > > rohit.yadav@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > > On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> > > wrote:
> > >
> > > > The following tests are running cleanly on KVM with Advanced
> networking
> > > > and 2 hosts.
> > > >
> > > > echo "Running tests with required_hardware=true"
> > > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > > tags=advanced,required_hardware=true \
> > > > smoke/test_password_server.py \
> > > > smoke/test_vpc_redundant.py \
> > > > smoke/test_routers_iptables_default_policy.py \
> > > > smoke/test_routers_network_ops.py \
> > > > smoke/test_vpc_router_nics.py \
> > > > smoke/test_router_dhcphosts.py \
> > > > smoke/test_loadbalance.py \
> > > > smoke/test_internal_lb.py \
> > > > smoke/test_ssvm.py \
> > > > smoke/test_vpc_vpn.py \
> > > > smoke/test_privategw_acl.py \
> > > > smoke/test_network.py
> > > >
> > > > echo "Running tests with required_hardware=false"
> > > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > > tags=advanced,required_hardware=false \
> > > > smoke/test_routers.py \
> > > > smoke/test_network_acl.py \
> > > > smoke/test_reset_vm_on_reboot.py \
> > > > smoke/test_vm_life_cycle.py \
> > > > smoke/test_service_offerings.py \
> > > > smoke/test_network.py \
> > > > component/test_vpc_offerings.py \
> > > > component/test_vpc_routers.py
> > > >
> > > > I need to do some more manual testing...
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > > w cloudops.com *|* tw @CloudOps_
> > > >
> > > > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > > > Mike.Tutkowski@netapp.com> wrote:
> > > >
> > > >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> > > >>
> > > >> I executed all of the tests in test/integration/plugins/solidfire.
> > > >>
> > > >> They all came back successful.
> > > >> ________________________________________
> > > >> From: John Burwell <jo...@shapeblue.com>
> > > >> Sent: Friday, June 17, 2016 12:56 PM
> > > >> To: dev@cloudstack.apache.org
> > > >> Subject: 4.9/master Testing Coordination
> > > >>
> > > >> All,
> > > >>
> > > >> It is a bit lo-fi, but if you are testing master in preparation for
> > the
> > > >> 4.9 RC, could you please share information about the configurations
> > you
> > > >> testing (e.g. hypervisors, storage backends, network configurations,
> > > etc)?
> > > >> Any test results could also be helpful.  The hope is to reduce
> > > duplication
> > > >> of effort and understand how much of the system has been covered.
> > > >>
> > > >> Thanks,
> > > >> -John
> > > >> john.burwell@shapeblue.com
> > > >> www.shapeblue.com<http://www.shapeblue.com>
> > > ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> > > www.shapeblue.com<http://www.shapeblue.com>
> > > Overview Apache CloudStack contains an authentication module providing
> > > "single sign-on" functionality via the SAML data format. Under certain
> > > conditions, a
> > >
> > >
> > >
> > > >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > > >> @shapeblue
> > > >>
> > > >>
> > > >>
> > > >
> > >
> >
>

Re: 4.9/master Testing Coordination

Posted by Will Stevens <ws...@cloudops.com>.
Great, thanks for the support Paul.  :)

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, Jun 29, 2016 at 10:12 AM, Paul Angus <pa...@shapeblue.com>
wrote:

> Thanks for the update Will.
> I'll look for the VR issue, I think that I've it in 4.5 intermittently on
> vmware. - in fact I've seen it come up with 6 interfaces before...
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: 29 June 2016 15:09
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> We are keeping an eye on the NIO SSH stuff because we have had reports of
> the connection being lost for more than 10 minutes (before manual
> intervention), but it seems intermittent.
>
> There is also a potential issue with the VR coming up with only two nics
> instead of three, but there are conflicting reports on that one, so keep an
> eye out for that too.
>
> Most of the details of the things people have come up against are in this
> thread.
>
> I will be camping till monday, so I will have very limited connectivity.
> I will hopefully cut the RC when I get back, so please let me know if you
> find anything this week.
>
> Thanks,
>
> Will
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Wed, Jun 29, 2016 at 9:15 AM, Paul Angus <pa...@shapeblue.com>
> wrote:
>
> > Hi Will,
> >
> > What's the current situation with the 4.9 release?
> > I'm helping John with setting up the environments which he's listed
> > for testing. Do we have a list of any blocker bugs that need clearing
> > or a timetable?
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -----Original Message-----
> > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: 28 June 2016 18:16
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Rohit,
> > Simon and his team is in the final push getting their next internal
> > production release ready to ship, so their testing of 4.9 has paused
> while
> > they get that ready.  I suspect he and his team will have limited time to
> > dedicate to this in this week.
> >
> > Just figured I would pass that along if he is not able to get back to you
> > quickly so you understand why there may be a delay.  :)
> >
> > Cheers,
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
> > wrote:
> >
> > > Hi Simon,
> > >
> > >
> > > Thanks for sharing. I'm continuing the discussion from a PR (
> > > https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> > > here wrt NIO ssl agent connection. Are you using latest master, and
> > > are you able to consistently reproduce the issue? Make sure that both
> > > management server and kvm-agent (or ssvm agents etc.) are using the
> > > packages built from latest master as well (i.e. updating both agent
> > > and mgmt server pkg jars is necessary).
> > >
> > >
> > > > ""I did also run into the NIO SSL agent not connecting problem again.
> > > When I telnetted to 8250, the agent immediately came up without me
> > > having to restart it. So keep an eye out for that as well.""
> > >
> > >
> > > When you ran into the issue, what was the environment like. Did you
> > > have to wait long enough before you decided to telnet on agent's port
> > > 8250? The current master's NioConnection code makes sure to check for
> > > new channels/connections every one second, this change ensures that
> > > reconnections are fast and non-blocking.
> > >
> > >
> > > Regards.
> > >
> > > ________________________________
> > > From: Simon Weller <sw...@ena.com>
> > > Sent: 18 June 2016 02:17:08
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: 4.9/master Testing Coordination
> > >
> > > Here's a quick run down on the configuration(s) we're testing:
> > >
> > >
> > > Centos 7.2
> > >
> > > Advanced Zone with VXLAN on KVM
> > >
> > > VPC functionality including Private GW, VPN, Static Routes, ACL Lists
> > > et al
> > >
> > > Redundant VPC VRs
> > >
> > > Ceph Primary Storage
> > >
> > > NFS and S3 secondary storage
> > >
> > > As Will mentioned, we've found an odd issue with VPCs that we're still
> > > debugging.
> > >
> > > Here's a summary of what we've found thus far:
> > > When a tier is added, the network interface for the tier network is
> > > never plugged by libvirt. You only get 2 interfaces (eth0 and eth1).
> > > eth2 is never plugged when you attempt to provision the first VM
> > > within the VPC and the VM creation fails. If you have existing VMs and
> > > you restart the router, you lose the eth2 interface in the libvirt
> > > configuration (confirmed with a virsh dumpxml).
> > > If you leave the VRs alone after the upgrade, VMs can be provisioned
> > > correctly (until you reboot the VRs).
> > >
> > > I did also run into the NIO SSL agent not connecting problem again.
> > > When I telnetted to 8250, the agent immediately came up without me
> > > having to restart it. So keep an eye out for that as well.
> > >
> > >
> > > - Si
> > >
> > > ________________________________
> > > From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> > > Will Stevens <ws...@cloudops.com>
> > > Sent: Friday, June 17, 2016 3:02 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: 4.9/master Testing Coordination
> > >
> > > Syed is looking into a potential issue with Swift as secondary storage
> > > on master.
> > >
> > > ENA is looking into a potential problem when upgrading a VR which was
> > > working in 4.8.0, but after an upgrade to 4.9.0 and restarting the
> > > network there are only 2 nics instead of 3.  If they spin a new VR
> > > from scratch it seems to work.  I need to follow up with them to see
> > > if they have an updated status of their testing.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > > @CloudOps_
> > >
> > >
> > > rohit.yadav@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > > On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> > > wrote:
> > >
> > > > The following tests are running cleanly on KVM with Advanced
> networking
> > > > and 2 hosts.
> > > >
> > > > echo "Running tests with required_hardware=true"
> > > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > > tags=advanced,required_hardware=true \
> > > > smoke/test_password_server.py \
> > > > smoke/test_vpc_redundant.py \
> > > > smoke/test_routers_iptables_default_policy.py \
> > > > smoke/test_routers_network_ops.py \
> > > > smoke/test_vpc_router_nics.py \
> > > > smoke/test_router_dhcphosts.py \
> > > > smoke/test_loadbalance.py \
> > > > smoke/test_internal_lb.py \
> > > > smoke/test_ssvm.py \
> > > > smoke/test_vpc_vpn.py \
> > > > smoke/test_privategw_acl.py \
> > > > smoke/test_network.py
> > > >
> > > > echo "Running tests with required_hardware=false"
> > > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > > tags=advanced,required_hardware=false \
> > > > smoke/test_routers.py \
> > > > smoke/test_network_acl.py \
> > > > smoke/test_reset_vm_on_reboot.py \
> > > > smoke/test_vm_life_cycle.py \
> > > > smoke/test_service_offerings.py \
> > > > smoke/test_network.py \
> > > > component/test_vpc_offerings.py \
> > > > component/test_vpc_routers.py
> > > >
> > > > I need to do some more manual testing...
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > > w cloudops.com *|* tw @CloudOps_
> > > >
> > > > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > > > Mike.Tutkowski@netapp.com> wrote:
> > > >
> > > >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> > > >>
> > > >> I executed all of the tests in test/integration/plugins/solidfire.
> > > >>
> > > >> They all came back successful.
> > > >> ________________________________________
> > > >> From: John Burwell <jo...@shapeblue.com>
> > > >> Sent: Friday, June 17, 2016 12:56 PM
> > > >> To: dev@cloudstack.apache.org
> > > >> Subject: 4.9/master Testing Coordination
> > > >>
> > > >> All,
> > > >>
> > > >> It is a bit lo-fi, but if you are testing master in preparation for
> > the
> > > >> 4.9 RC, could you please share information about the configurations
> > you
> > > >> testing (e.g. hypervisors, storage backends, network configurations,
> > > etc)?
> > > >> Any test results could also be helpful.  The hope is to reduce
> > > duplication
> > > >> of effort and understand how much of the system has been covered.
> > > >>
> > > >> Thanks,
> > > >> -John
> > > >> john.burwell@shapeblue.com
> > > >> www.shapeblue.com<http://www.shapeblue.com>
> > > ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> > > www.shapeblue.com<http://www.shapeblue.com>
> > > Overview Apache CloudStack contains an authentication module providing
> > > "single sign-on" functionality via the SAML data format. Under certain
> > > conditions, a
> > >
> > >
> > >
> > > >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > > >> @shapeblue
> > > >>
> > > >>
> > > >>
> > > >
> > >
> >
>

RE: 4.9/master Testing Coordination

Posted by Paul Angus <pa...@shapeblue.com>.
Thanks for the update Will.
I'll look for the VR issue, I think that I've it in 4.5 intermittently on vmware. - in fact I've seen it come up with 6 interfaces before...


Kind regards,

Paul Angus

paul.angus@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: 29 June 2016 15:09
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

We are keeping an eye on the NIO SSH stuff because we have had reports of the connection being lost for more than 10 minutes (before manual intervention), but it seems intermittent.

There is also a potential issue with the VR coming up with only two nics instead of three, but there are conflicting reports on that one, so keep an eye out for that too.

Most of the details of the things people have come up against are in this thread.

I will be camping till monday, so I will have very limited connectivity.  I will hopefully cut the RC when I get back, so please let me know if you find anything this week.

Thanks,

Will

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_

On Wed, Jun 29, 2016 at 9:15 AM, Paul Angus <pa...@shapeblue.com>
wrote:

> Hi Will,
>
> What's the current situation with the 4.9 release?
> I'm helping John with setting up the environments which he's listed 
> for testing. Do we have a list of any blocker bugs that need clearing 
> or a timetable?
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: 28 June 2016 18:16
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Rohit,
> Simon and his team is in the final push getting their next internal
> production release ready to ship, so their testing of 4.9 has paused while
> they get that ready.  I suspect he and his team will have limited time to
> dedicate to this in this week.
>
> Just figured I would pass that along if he is not able to get back to you
> quickly so you understand why there may be a delay.  :)
>
> Cheers,
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
> wrote:
>
> > Hi Simon,
> >
> >
> > Thanks for sharing. I'm continuing the discussion from a PR (
> > https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> > here wrt NIO ssl agent connection. Are you using latest master, and
> > are you able to consistently reproduce the issue? Make sure that both
> > management server and kvm-agent (or ssvm agents etc.) are using the
> > packages built from latest master as well (i.e. updating both agent
> > and mgmt server pkg jars is necessary).
> >
> >
> > > ""I did also run into the NIO SSL agent not connecting problem again.
> > When I telnetted to 8250, the agent immediately came up without me
> > having to restart it. So keep an eye out for that as well.""
> >
> >
> > When you ran into the issue, what was the environment like. Did you
> > have to wait long enough before you decided to telnet on agent's port
> > 8250? The current master's NioConnection code makes sure to check for
> > new channels/connections every one second, this change ensures that
> > reconnections are fast and non-blocking.
> >
> >
> > Regards.
> >
> > ________________________________
> > From: Simon Weller <sw...@ena.com>
> > Sent: 18 June 2016 02:17:08
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Here's a quick run down on the configuration(s) we're testing:
> >
> >
> > Centos 7.2
> >
> > Advanced Zone with VXLAN on KVM
> >
> > VPC functionality including Private GW, VPN, Static Routes, ACL Lists
> > et al
> >
> > Redundant VPC VRs
> >
> > Ceph Primary Storage
> >
> > NFS and S3 secondary storage
> >
> > As Will mentioned, we've found an odd issue with VPCs that we're still
> > debugging.
> >
> > Here's a summary of what we've found thus far:
> > When a tier is added, the network interface for the tier network is
> > never plugged by libvirt. You only get 2 interfaces (eth0 and eth1).
> > eth2 is never plugged when you attempt to provision the first VM
> > within the VPC and the VM creation fails. If you have existing VMs and
> > you restart the router, you lose the eth2 interface in the libvirt
> > configuration (confirmed with a virsh dumpxml).
> > If you leave the VRs alone after the upgrade, VMs can be provisioned
> > correctly (until you reboot the VRs).
> >
> > I did also run into the NIO SSL agent not connecting problem again.
> > When I telnetted to 8250, the agent immediately came up without me
> > having to restart it. So keep an eye out for that as well.
> >
> >
> > - Si
> >
> > ________________________________
> > From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> > Will Stevens <ws...@cloudops.com>
> > Sent: Friday, June 17, 2016 3:02 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Syed is looking into a potential issue with Swift as secondary storage
> > on master.
> >
> > ENA is looking into a potential problem when upgrading a VR which was
> > working in 4.8.0, but after an upgrade to 4.9.0 and restarting the
> > network there are only 2 nics instead of 3.  If they spin a new VR
> > from scratch it seems to work.  I need to follow up with them to see
> > if they have an updated status of their testing.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> >
> > rohit.yadav@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> > On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > The following tests are running cleanly on KVM with Advanced networking
> > > and 2 hosts.
> > >
> > > echo "Running tests with required_hardware=true"
> > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > tags=advanced,required_hardware=true \
> > > smoke/test_password_server.py \
> > > smoke/test_vpc_redundant.py \
> > > smoke/test_routers_iptables_default_policy.py \
> > > smoke/test_routers_network_ops.py \
> > > smoke/test_vpc_router_nics.py \
> > > smoke/test_router_dhcphosts.py \
> > > smoke/test_loadbalance.py \
> > > smoke/test_internal_lb.py \
> > > smoke/test_ssvm.py \
> > > smoke/test_vpc_vpn.py \
> > > smoke/test_privategw_acl.py \
> > > smoke/test_network.py
> > >
> > > echo "Running tests with required_hardware=false"
> > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > tags=advanced,required_hardware=false \
> > > smoke/test_routers.py \
> > > smoke/test_network_acl.py \
> > > smoke/test_reset_vm_on_reboot.py \
> > > smoke/test_vm_life_cycle.py \
> > > smoke/test_service_offerings.py \
> > > smoke/test_network.py \
> > > component/test_vpc_offerings.py \
> > > component/test_vpc_routers.py
> > >
> > > I need to do some more manual testing...
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > w cloudops.com *|* tw @CloudOps_
> > >
> > > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > > Mike.Tutkowski@netapp.com> wrote:
> > >
> > >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> > >>
> > >> I executed all of the tests in test/integration/plugins/solidfire.
> > >>
> > >> They all came back successful.
> > >> ________________________________________
> > >> From: John Burwell <jo...@shapeblue.com>
> > >> Sent: Friday, June 17, 2016 12:56 PM
> > >> To: dev@cloudstack.apache.org
> > >> Subject: 4.9/master Testing Coordination
> > >>
> > >> All,
> > >>
> > >> It is a bit lo-fi, but if you are testing master in preparation for
> the
> > >> 4.9 RC, could you please share information about the configurations
> you
> > >> testing (e.g. hypervisors, storage backends, network configurations,
> > etc)?
> > >> Any test results could also be helpful.  The hope is to reduce
> > duplication
> > >> of effort and understand how much of the system has been covered.
> > >>
> > >> Thanks,
> > >> -John
> > >> john.burwell@shapeblue.com
> > >> www.shapeblue.com<http://www.shapeblue.com>
> > ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> > www.shapeblue.com<http://www.shapeblue.com>
> > Overview Apache CloudStack contains an authentication module providing
> > "single sign-on" functionality via the SAML data format. Under certain
> > conditions, a
> >
> >
> >
> > >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > >> @shapeblue
> > >>
> > >>
> > >>
> > >
> >
>

Re: 4.9/master Testing Coordination

Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Will,


I've created a PR that aims to make Nio related connections slightly aggressive which should solve any timeout/reconnection issues. If you're still seeing the issue with *latest* master, please test and merge this PR https://github.com/apache/cloudstack/pull/1601

I could not reproduce the issue at all, in all cases it worked without an issue in my environment. Please, do let me know if hit the issue at all in your environment.


Regards.

________________________________
From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
Sent: 29 June 2016 19:39:00
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

We are keeping an eye on the NIO SSH stuff because we have had reports of
the connection being lost for more than 10 minutes (before manual
intervention), but it seems intermittent.

There is also a potential issue with the VR coming up with only two nics
instead of three, but there are conflicting reports on that one, so keep an
eye out for that too.

Most of the details of the things people have come up against are in this
thread.

I will be camping till monday, so I will have very limited connectivity.  I
will hopefully cut the RC when I get back, so please let me know if you
find anything this week.

Thanks,

Will

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, Jun 29, 2016 at 9:15 AM, Paul Angus <pa...@shapeblue.com>
wrote:

> Hi Will,
>
> What's the current situation with the 4.9 release?
> I'm helping John with setting up the environments which he's listed for
> testing. Do we have a list of any blocker bugs that need clearing or a
> timetable?
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: 28 June 2016 18:16
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Rohit,
> Simon and his team is in the final push getting their next internal
> production release ready to ship, so their testing of 4.9 has paused while
> they get that ready.  I suspect he and his team will have limited time to
> dedicate to this in this week.
>
> Just figured I would pass that along if he is not able to get back to you
> quickly so you understand why there may be a delay.  :)
>
> Cheers,
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
> wrote:
>
> > Hi Simon,
> >
> >
> > Thanks for sharing. I'm continuing the discussion from a PR (
> > https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> > here wrt NIO ssl agent connection. Are you using latest master, and
> > are you able to consistently reproduce the issue? Make sure that both
> > management server and kvm-agent (or ssvm agents etc.) are using the
> > packages built from latest master as well (i.e. updating both agent
> > and mgmt server pkg jars is necessary).
> >
> >
> > > ""I did also run into the NIO SSL agent not connecting problem again.
> > When I telnetted to 8250, the agent immediately came up without me
> > having to restart it. So keep an eye out for that as well.""
> >
> >
> > When you ran into the issue, what was the environment like. Did you
> > have to wait long enough before you decided to telnet on agent's port
> > 8250? The current master's NioConnection code makes sure to check for
> > new channels/connections every one second, this change ensures that
> > reconnections are fast and non-blocking.
> >
> >
> > Regards.
> >
> > ________________________________
> > From: Simon Weller <sw...@ena.com>
> > Sent: 18 June 2016 02:17:08
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Here's a quick run down on the configuration(s) we're testing:
> >
> >
> > Centos 7.2
> >
> > Advanced Zone with VXLAN on KVM
> >
> > VPC functionality including Private GW, VPN, Static Routes, ACL Lists
> > et al
> >
> > Redundant VPC VRs
> >
> > Ceph Primary Storage
> >
> > NFS and S3 secondary storage
> >
> > As Will mentioned, we've found an odd issue with VPCs that we're still
> > debugging.
> >
> > Here's a summary of what we've found thus far:
> > When a tier is added, the network interface for the tier network is
> > never plugged by libvirt. You only get 2 interfaces (eth0 and eth1).
> > eth2 is never plugged when you attempt to provision the first VM
> > within the VPC and the VM creation fails. If you have existing VMs and
> > you restart the router, you lose the eth2 interface in the libvirt
> > configuration (confirmed with a virsh dumpxml).
> > If you leave the VRs alone after the upgrade, VMs can be provisioned
> > correctly (until you reboot the VRs).
> >
> > I did also run into the NIO SSL agent not connecting problem again.
> > When I telnetted to 8250, the agent immediately came up without me
> > having to restart it. So keep an eye out for that as well.
> >
> >
> > - Si
> >
> > ________________________________
> > From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> > Will Stevens <ws...@cloudops.com>
> > Sent: Friday, June 17, 2016 3:02 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Syed is looking into a potential issue with Swift as secondary storage
> > on master.
> >
> > ENA is looking into a potential problem when upgrading a VR which was
> > working in 4.8.0, but after an upgrade to 4.9.0 and restarting the
> > network there are only 2 nics instead of 3.  If they spin a new VR
> > from scratch it seems to work.  I need to follow up with them to see
> > if they have an updated status of their testing.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> >
> > rohit.yadav@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> > On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > The following tests are running cleanly on KVM with Advanced networking
> > > and 2 hosts.
> > >
> > > echo "Running tests with required_hardware=true"
> > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > tags=advanced,required_hardware=true \
> > > smoke/test_password_server.py \
> > > smoke/test_vpc_redundant.py \
> > > smoke/test_routers_iptables_default_policy.py \
> > > smoke/test_routers_network_ops.py \
> > > smoke/test_vpc_router_nics.py \
> > > smoke/test_router_dhcphosts.py \
> > > smoke/test_loadbalance.py \
> > > smoke/test_internal_lb.py \
> > > smoke/test_ssvm.py \
> > > smoke/test_vpc_vpn.py \
> > > smoke/test_privategw_acl.py \
> > > smoke/test_network.py
> > >
> > > echo "Running tests with required_hardware=false"
> > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > tags=advanced,required_hardware=false \
> > > smoke/test_routers.py \
> > > smoke/test_network_acl.py \
> > > smoke/test_reset_vm_on_reboot.py \
> > > smoke/test_vm_life_cycle.py \
> > > smoke/test_service_offerings.py \
> > > smoke/test_network.py \
> > > component/test_vpc_offerings.py \
> > > component/test_vpc_routers.py
> > >
> > > I need to do some more manual testing...
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > w cloudops.com *|* tw @CloudOps_
> > >
> > > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > > Mike.Tutkowski@netapp.com> wrote:
> > >
> > >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> > >>
> > >> I executed all of the tests in test/integration/plugins/solidfire.
> > >>
> > >> They all came back successful.
> > >> ________________________________________
> > >> From: John Burwell <jo...@shapeblue.com>
> > >> Sent: Friday, June 17, 2016 12:56 PM
> > >> To: dev@cloudstack.apache.org
> > >> Subject: 4.9/master Testing Coordination
> > >>
> > >> All,
> > >>
> > >> It is a bit lo-fi, but if you are testing master in preparation for
> the
> > >> 4.9 RC, could you please share information about the configurations
> you
> > >> testing (e.g. hypervisors, storage backends, network configurations,
> > etc)?
> > >> Any test results could also be helpful.  The hope is to reduce
> > duplication
> > >> of effort and understand how much of the system has been covered.
> > >>
> > >> Thanks,
> > >> -John
> > >> john.burwell@shapeblue.com
> > >> www.shapeblue.com<http://www.shapeblue.com>
> > ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> > www.shapeblue.com<http://www.shapeblue.com>
> > Overview Apache CloudStack contains an authentication module providing
> > "single sign-on" functionality via the SAML data format. Under certain
> > conditions, a
> >
> >
> >
> > >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > >> @shapeblue
> > >>
> > >>
> > >>
> > >
> >
>

rohit.yadav@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


Re: 4.9/master Testing Coordination

Posted by Will Stevens <ws...@cloudops.com>.
We are keeping an eye on the NIO SSH stuff because we have had reports of
the connection being lost for more than 10 minutes (before manual
intervention), but it seems intermittent.

There is also a potential issue with the VR coming up with only two nics
instead of three, but there are conflicting reports on that one, so keep an
eye out for that too.

Most of the details of the things people have come up against are in this
thread.

I will be camping till monday, so I will have very limited connectivity.  I
will hopefully cut the RC when I get back, so please let me know if you
find anything this week.

Thanks,

Will

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, Jun 29, 2016 at 9:15 AM, Paul Angus <pa...@shapeblue.com>
wrote:

> Hi Will,
>
> What's the current situation with the 4.9 release?
> I'm helping John with setting up the environments which he's listed for
> testing. Do we have a list of any blocker bugs that need clearing or a
> timetable?
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: 28 June 2016 18:16
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Rohit,
> Simon and his team is in the final push getting their next internal
> production release ready to ship, so their testing of 4.9 has paused while
> they get that ready.  I suspect he and his team will have limited time to
> dedicate to this in this week.
>
> Just figured I would pass that along if he is not able to get back to you
> quickly so you understand why there may be a delay.  :)
>
> Cheers,
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
> wrote:
>
> > Hi Simon,
> >
> >
> > Thanks for sharing. I'm continuing the discussion from a PR (
> > https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> > here wrt NIO ssl agent connection. Are you using latest master, and
> > are you able to consistently reproduce the issue? Make sure that both
> > management server and kvm-agent (or ssvm agents etc.) are using the
> > packages built from latest master as well (i.e. updating both agent
> > and mgmt server pkg jars is necessary).
> >
> >
> > > ""I did also run into the NIO SSL agent not connecting problem again.
> > When I telnetted to 8250, the agent immediately came up without me
> > having to restart it. So keep an eye out for that as well.""
> >
> >
> > When you ran into the issue, what was the environment like. Did you
> > have to wait long enough before you decided to telnet on agent's port
> > 8250? The current master's NioConnection code makes sure to check for
> > new channels/connections every one second, this change ensures that
> > reconnections are fast and non-blocking.
> >
> >
> > Regards.
> >
> > ________________________________
> > From: Simon Weller <sw...@ena.com>
> > Sent: 18 June 2016 02:17:08
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Here's a quick run down on the configuration(s) we're testing:
> >
> >
> > Centos 7.2
> >
> > Advanced Zone with VXLAN on KVM
> >
> > VPC functionality including Private GW, VPN, Static Routes, ACL Lists
> > et al
> >
> > Redundant VPC VRs
> >
> > Ceph Primary Storage
> >
> > NFS and S3 secondary storage
> >
> > As Will mentioned, we've found an odd issue with VPCs that we're still
> > debugging.
> >
> > Here's a summary of what we've found thus far:
> > When a tier is added, the network interface for the tier network is
> > never plugged by libvirt. You only get 2 interfaces (eth0 and eth1).
> > eth2 is never plugged when you attempt to provision the first VM
> > within the VPC and the VM creation fails. If you have existing VMs and
> > you restart the router, you lose the eth2 interface in the libvirt
> > configuration (confirmed with a virsh dumpxml).
> > If you leave the VRs alone after the upgrade, VMs can be provisioned
> > correctly (until you reboot the VRs).
> >
> > I did also run into the NIO SSL agent not connecting problem again.
> > When I telnetted to 8250, the agent immediately came up without me
> > having to restart it. So keep an eye out for that as well.
> >
> >
> > - Si
> >
> > ________________________________
> > From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> > Will Stevens <ws...@cloudops.com>
> > Sent: Friday, June 17, 2016 3:02 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: 4.9/master Testing Coordination
> >
> > Syed is looking into a potential issue with Swift as secondary storage
> > on master.
> >
> > ENA is looking into a potential problem when upgrading a VR which was
> > working in 4.8.0, but after an upgrade to 4.9.0 and restarting the
> > network there are only 2 nics instead of 3.  If they spin a new VR
> > from scratch it seems to work.  I need to follow up with them to see
> > if they have an updated status of their testing.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> >
> > rohit.yadav@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> > On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > The following tests are running cleanly on KVM with Advanced networking
> > > and 2 hosts.
> > >
> > > echo "Running tests with required_hardware=true"
> > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > tags=advanced,required_hardware=true \
> > > smoke/test_password_server.py \
> > > smoke/test_vpc_redundant.py \
> > > smoke/test_routers_iptables_default_policy.py \
> > > smoke/test_routers_network_ops.py \
> > > smoke/test_vpc_router_nics.py \
> > > smoke/test_router_dhcphosts.py \
> > > smoke/test_loadbalance.py \
> > > smoke/test_internal_lb.py \
> > > smoke/test_ssvm.py \
> > > smoke/test_vpc_vpn.py \
> > > smoke/test_privategw_acl.py \
> > > smoke/test_network.py
> > >
> > > echo "Running tests with required_hardware=false"
> > > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > > tags=advanced,required_hardware=false \
> > > smoke/test_routers.py \
> > > smoke/test_network_acl.py \
> > > smoke/test_reset_vm_on_reboot.py \
> > > smoke/test_vm_life_cycle.py \
> > > smoke/test_service_offerings.py \
> > > smoke/test_network.py \
> > > component/test_vpc_offerings.py \
> > > component/test_vpc_routers.py
> > >
> > > I need to do some more manual testing...
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > w cloudops.com *|* tw @CloudOps_
> > >
> > > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > > Mike.Tutkowski@netapp.com> wrote:
> > >
> > >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> > >>
> > >> I executed all of the tests in test/integration/plugins/solidfire.
> > >>
> > >> They all came back successful.
> > >> ________________________________________
> > >> From: John Burwell <jo...@shapeblue.com>
> > >> Sent: Friday, June 17, 2016 12:56 PM
> > >> To: dev@cloudstack.apache.org
> > >> Subject: 4.9/master Testing Coordination
> > >>
> > >> All,
> > >>
> > >> It is a bit lo-fi, but if you are testing master in preparation for
> the
> > >> 4.9 RC, could you please share information about the configurations
> you
> > >> testing (e.g. hypervisors, storage backends, network configurations,
> > etc)?
> > >> Any test results could also be helpful.  The hope is to reduce
> > duplication
> > >> of effort and understand how much of the system has been covered.
> > >>
> > >> Thanks,
> > >> -John
> > >> john.burwell@shapeblue.com
> > >> www.shapeblue.com<http://www.shapeblue.com>
> > ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> > www.shapeblue.com<http://www.shapeblue.com>
> > Overview Apache CloudStack contains an authentication module providing
> > "single sign-on" functionality via the SAML data format. Under certain
> > conditions, a
> >
> >
> >
> > >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > >> @shapeblue
> > >>
> > >>
> > >>
> > >
> >
>

RE: 4.9/master Testing Coordination

Posted by Paul Angus <pa...@shapeblue.com>.
Hi Will,

What's the current situation with the 4.9 release?  
I'm helping John with setting up the environments which he's listed for testing. Do we have a list of any blocker bugs that need clearing or a timetable?

Kind regards,

Paul Angus

paul.angus@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: 28 June 2016 18:16
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

Rohit,
Simon and his team is in the final push getting their next internal production release ready to ship, so their testing of 4.9 has paused while they get that ready.  I suspect he and his team will have limited time to dedicate to this in this week.

Just figured I would pass that along if he is not able to get back to you quickly so you understand why there may be a delay.  :)

Cheers,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_

On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
wrote:

> Hi Simon,
>
>
> Thanks for sharing. I'm continuing the discussion from a PR (
> https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> here wrt NIO ssl agent connection. Are you using latest master, and 
> are you able to consistently reproduce the issue? Make sure that both 
> management server and kvm-agent (or ssvm agents etc.) are using the 
> packages built from latest master as well (i.e. updating both agent 
> and mgmt server pkg jars is necessary).
>
>
> > ""I did also run into the NIO SSL agent not connecting problem again.
> When I telnetted to 8250, the agent immediately came up without me 
> having to restart it. So keep an eye out for that as well.""
>
>
> When you ran into the issue, what was the environment like. Did you 
> have to wait long enough before you decided to telnet on agent's port 
> 8250? The current master's NioConnection code makes sure to check for 
> new channels/connections every one second, this change ensures that 
> reconnections are fast and non-blocking.
>
>
> Regards.
>
> ________________________________
> From: Simon Weller <sw...@ena.com>
> Sent: 18 June 2016 02:17:08
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Here's a quick run down on the configuration(s) we're testing:
>
>
> Centos 7.2
>
> Advanced Zone with VXLAN on KVM
>
> VPC functionality including Private GW, VPN, Static Routes, ACL Lists 
> et al
>
> Redundant VPC VRs
>
> Ceph Primary Storage
>
> NFS and S3 secondary storage
>
> As Will mentioned, we've found an odd issue with VPCs that we're still 
> debugging.
>
> Here's a summary of what we've found thus far:
> When a tier is added, the network interface for the tier network is 
> never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). 
> eth2 is never plugged when you attempt to provision the first VM 
> within the VPC and the VM creation fails. If you have existing VMs and 
> you restart the router, you lose the eth2 interface in the libvirt 
> configuration (confirmed with a virsh dumpxml).
> If you leave the VRs alone after the upgrade, VMs can be provisioned 
> correctly (until you reboot the VRs).
>
> I did also run into the NIO SSL agent not connecting problem again. 
> When I telnetted to 8250, the agent immediately came up without me 
> having to restart it. So keep an eye out for that as well.
>
>
> - Si
>
> ________________________________
> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of 
> Will Stevens <ws...@cloudops.com>
> Sent: Friday, June 17, 2016 3:02 PM
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Syed is looking into a potential issue with Swift as secondary storage 
> on master.
>
> ENA is looking into a potential problem when upgrading a VR which was 
> working in 4.8.0, but after an upgrade to 4.9.0 and restarting the 
> network there are only 2 nics instead of 3.  If they spin a new VR 
> from scratch it seems to work.  I need to follow up with them to see 
> if they have an updated status of their testing.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw 
> @CloudOps_
>
>
> rohit.yadav@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
> On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > The following tests are running cleanly on KVM with Advanced networking
> > and 2 hosts.
> >
> > echo "Running tests with required_hardware=true"
> > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > tags=advanced,required_hardware=true \
> > smoke/test_password_server.py \
> > smoke/test_vpc_redundant.py \
> > smoke/test_routers_iptables_default_policy.py \
> > smoke/test_routers_network_ops.py \
> > smoke/test_vpc_router_nics.py \
> > smoke/test_router_dhcphosts.py \
> > smoke/test_loadbalance.py \
> > smoke/test_internal_lb.py \
> > smoke/test_ssvm.py \
> > smoke/test_vpc_vpn.py \
> > smoke/test_privategw_acl.py \
> > smoke/test_network.py
> >
> > echo "Running tests with required_hardware=false"
> > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > tags=advanced,required_hardware=false \
> > smoke/test_routers.py \
> > smoke/test_network_acl.py \
> > smoke/test_reset_vm_on_reboot.py \
> > smoke/test_vm_life_cycle.py \
> > smoke/test_service_offerings.py \
> > smoke/test_network.py \
> > component/test_vpc_offerings.py \
> > component/test_vpc_routers.py
> >
> > I need to do some more manual testing...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > Mike.Tutkowski@netapp.com> wrote:
> >
> >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> >>
> >> I executed all of the tests in test/integration/plugins/solidfire.
> >>
> >> They all came back successful.
> >> ________________________________________
> >> From: John Burwell <jo...@shapeblue.com>
> >> Sent: Friday, June 17, 2016 12:56 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: 4.9/master Testing Coordination
> >>
> >> All,
> >>
> >> It is a bit lo-fi, but if you are testing master in preparation for the
> >> 4.9 RC, could you please share information about the configurations you
> >> testing (e.g. hypervisors, storage backends, network configurations,
> etc)?
> >> Any test results could also be helpful.  The hope is to reduce
> duplication
> >> of effort and understand how much of the system has been covered.
> >>
> >> Thanks,
> >> -John
> >> john.burwell@shapeblue.com
> >> www.shapeblue.com<http://www.shapeblue.com>
> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> www.shapeblue.com<http://www.shapeblue.com>
> Overview Apache CloudStack contains an authentication module providing
> "single sign-on" functionality via the SAML data format. Under certain
> conditions, a
>
>
>
> >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> >> @shapeblue
> >>
> >>
> >>
> >
>

Re: 4.9/master Testing Coordination

Posted by Will Stevens <ws...@cloudops.com>.
Rohit,
Simon and his team is in the final push getting their next internal
production release ready to ship, so their testing of 4.9 has paused while
they get that ready.  I suspect he and his team will have limited time to
dedicate to this in this week.

Just figured I would pass that along if he is not able to get back to you
quickly so you understand why there may be a delay.  :)

Cheers,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Tue, Jun 28, 2016 at 1:08 PM, Rohit Yadav <ro...@shapeblue.com>
wrote:

> Hi Simon,
>
>
> Thanks for sharing. I'm continuing the discussion from a PR (
> https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969)
> here wrt NIO ssl agent connection. Are you using latest master, and are you
> able to consistently reproduce the issue? Make sure that both management
> server and kvm-agent (or ssvm agents etc.) are using the packages built
> from latest master as well (i.e. updating both agent and mgmt server pkg
> jars is necessary).
>
>
> > ""I did also run into the NIO SSL agent not connecting problem again.
> When I telnetted to 8250, the agent immediately came up without me having
> to restart it. So keep an eye out for that as well.""
>
>
> When you ran into the issue, what was the environment like. Did you have
> to wait long enough before you decided to telnet on agent's port 8250? The
> current master's NioConnection code makes sure to check for new
> channels/connections every one second, this change ensures that
> reconnections are fast and non-blocking.
>
>
> Regards.
>
> ________________________________
> From: Simon Weller <sw...@ena.com>
> Sent: 18 June 2016 02:17:08
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Here's a quick run down on the configuration(s) we're testing:
>
>
> Centos 7.2
>
> Advanced Zone with VXLAN on KVM
>
> VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al
>
> Redundant VPC VRs
>
> Ceph Primary Storage
>
> NFS and S3 secondary storage
>
> As Will mentioned, we've found an odd issue with VPCs that we're still
> debugging.
>
> Here's a summary of what we've found thus far:
> When a tier is added, the network interface for the tier network is never
> plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is
> never plugged when you attempt to provision the first VM within the VPC and
> the VM creation fails. If you have existing VMs and you restart the router,
> you lose the eth2 interface in the libvirt configuration (confirmed with a
> virsh dumpxml).
> If you leave the VRs alone after the upgrade, VMs can be provisioned
> correctly (until you reboot the VRs).
>
> I did also run into the NIO SSL agent not connecting problem again. When I
> telnetted to 8250, the agent immediately came up without me having to
> restart it. So keep an eye out for that as well.
>
>
> - Si
>
> ________________________________
> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> Will Stevens <ws...@cloudops.com>
> Sent: Friday, June 17, 2016 3:02 PM
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9/master Testing Coordination
>
> Syed is looking into a potential issue with Swift as secondary storage on
> master.
>
> ENA is looking into a potential problem when upgrading a VR which was
> working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
> there are only 2 nics instead of 3.  If they spin a new VR from scratch it
> seems to work.  I need to follow up with them to see if they have an
> updated status of their testing.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
>
> rohit.yadav@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > The following tests are running cleanly on KVM with Advanced networking
> > and 2 hosts.
> >
> > echo "Running tests with required_hardware=true"
> > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > tags=advanced,required_hardware=true \
> > smoke/test_password_server.py \
> > smoke/test_vpc_redundant.py \
> > smoke/test_routers_iptables_default_policy.py \
> > smoke/test_routers_network_ops.py \
> > smoke/test_vpc_router_nics.py \
> > smoke/test_router_dhcphosts.py \
> > smoke/test_loadbalance.py \
> > smoke/test_internal_lb.py \
> > smoke/test_ssvm.py \
> > smoke/test_vpc_vpn.py \
> > smoke/test_privategw_acl.py \
> > smoke/test_network.py
> >
> > echo "Running tests with required_hardware=false"
> > nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> > tags=advanced,required_hardware=false \
> > smoke/test_routers.py \
> > smoke/test_network_acl.py \
> > smoke/test_reset_vm_on_reboot.py \
> > smoke/test_vm_life_cycle.py \
> > smoke/test_service_offerings.py \
> > smoke/test_network.py \
> > component/test_vpc_offerings.py \
> > component/test_vpc_routers.py
> >
> > I need to do some more manual testing...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> > Mike.Tutkowski@netapp.com> wrote:
> >
> >> My testing has been performed using XenServer 6.5 and ESXi 5.5.
> >>
> >> I executed all of the tests in test/integration/plugins/solidfire.
> >>
> >> They all came back successful.
> >> ________________________________________
> >> From: John Burwell <jo...@shapeblue.com>
> >> Sent: Friday, June 17, 2016 12:56 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: 4.9/master Testing Coordination
> >>
> >> All,
> >>
> >> It is a bit lo-fi, but if you are testing master in preparation for the
> >> 4.9 RC, could you please share information about the configurations you
> >> testing (e.g. hypervisors, storage backends, network configurations,
> etc)?
> >> Any test results could also be helpful.  The hope is to reduce
> duplication
> >> of effort and understand how much of the system has been covered.
> >>
> >> Thanks,
> >> -John
> >> john.burwell@shapeblue.com
> >> www.shapeblue.com<http://www.shapeblue.com>
> ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
> www.shapeblue.com<http://www.shapeblue.com>
> Overview Apache CloudStack contains an authentication module providing
> "single sign-on" functionality via the SAML data format. Under certain
> conditions, a
>
>
>
> >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> >> @shapeblue
> >>
> >>
> >>
> >
>

Re: 4.9/master Testing Coordination

Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Simon,


Thanks for sharing. I'm continuing the discussion from a PR (https://github.com/apache/cloudstack/pull/1549#issuecomment-229070969) here wrt NIO ssl agent connection. Are you using latest master, and are you able to consistently reproduce the issue? Make sure that both management server and kvm-agent (or ssvm agents etc.) are using the packages built from latest master as well (i.e. updating both agent and mgmt server pkg jars is necessary).


> ""I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.""


When you ran into the issue, what was the environment like. Did you have to wait long enough before you decided to telnet on agent's port 8250? The current master's NioConnection code makes sure to check for new channels/connections every one second, this change ensures that reconnections are fast and non-blocking.


Regards.

________________________________
From: Simon Weller <sw...@ena.com>
Sent: 18 June 2016 02:17:08
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

Here's a quick run down on the configuration(s) we're testing:


Centos 7.2

Advanced Zone with VXLAN on KVM

VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al

Redundant VPC VRs

Ceph Primary Storage

NFS and S3 secondary storage

As Will mentioned, we've found an odd issue with VPCs that we're still debugging.

Here's a summary of what we've found thus far:
When a tier is added, the network interface for the tier network is never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is never plugged when you attempt to provision the first VM within the VPC and the VM creation fails. If you have existing VMs and you restart the router, you lose the eth2 interface in the libvirt configuration (confirmed with a virsh dumpxml).
If you leave the VRs alone after the upgrade, VMs can be provisioned correctly (until you reboot the VRs).

I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.


- Si

________________________________
From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
Sent: Friday, June 17, 2016 3:02 PM
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

Syed is looking into a potential issue with Swift as secondary storage on
master.

ENA is looking into a potential problem when upgrading a VR which was
working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
there are only 2 nics instead of 3.  If they spin a new VR from scratch it
seems to work.  I need to follow up with them to see if they have an
updated status of their testing.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_


rohit.yadav@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:

> The following tests are running cleanly on KVM with Advanced networking
> and 2 hosts.
>
> echo "Running tests with required_hardware=true"
> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> tags=advanced,required_hardware=true \
> smoke/test_password_server.py \
> smoke/test_vpc_redundant.py \
> smoke/test_routers_iptables_default_policy.py \
> smoke/test_routers_network_ops.py \
> smoke/test_vpc_router_nics.py \
> smoke/test_router_dhcphosts.py \
> smoke/test_loadbalance.py \
> smoke/test_internal_lb.py \
> smoke/test_ssvm.py \
> smoke/test_vpc_vpn.py \
> smoke/test_privategw_acl.py \
> smoke/test_network.py
>
> echo "Running tests with required_hardware=false"
> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> tags=advanced,required_hardware=false \
> smoke/test_routers.py \
> smoke/test_network_acl.py \
> smoke/test_reset_vm_on_reboot.py \
> smoke/test_vm_life_cycle.py \
> smoke/test_service_offerings.py \
> smoke/test_network.py \
> component/test_vpc_offerings.py \
> component/test_vpc_routers.py
>
> I need to do some more manual testing...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> Mike.Tutkowski@netapp.com> wrote:
>
>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>
>> I executed all of the tests in test/integration/plugins/solidfire.
>>
>> They all came back successful.
>> ________________________________________
>> From: John Burwell <jo...@shapeblue.com>
>> Sent: Friday, June 17, 2016 12:56 PM
>> To: dev@cloudstack.apache.org
>> Subject: 4.9/master Testing Coordination
>>
>> All,
>>
>> It is a bit lo-fi, but if you are testing master in preparation for the
>> 4.9 RC, could you please share information about the configurations you
>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>> Any test results could also be helpful.  The hope is to reduce duplication
>> of effort and understand how much of the system has been covered.
>>
>> Thanks,
>> -John
>> john.burwell@shapeblue.com
>> www.shapeblue.com<http://www.shapeblue.com>
ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
www.shapeblue.com<http://www.shapeblue.com>
Overview Apache CloudStack contains an authentication module providing "single sign-on" functionality via the SAML data format. Under certain conditions, a



>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>

Re: 4.9/master Testing Coordination

Posted by Simon Weller <sw...@ena.com>.
Here's a quick run down on the configuration(s) we're testing:


Centos 7.2

Advanced Zone with VXLAN on KVM

VPC functionality including Private GW, VPN, Static Routes, ACL Lists et al

Redundant VPC VRs

Ceph Primary Storage

NFS and S3 secondary storage

As Will mentioned, we've found an odd issue with VPCs that we're still debugging.

Here's a summary of what we've found thus far:
When a tier is added, the network interface for the tier network is never plugged by libvirt. You only get 2 interfaces (eth0 and eth1). eth2 is never plugged when you attempt to provision the first VM within the VPC and the VM creation fails. If you have existing VMs and you restart the router, you lose the eth2 interface in the libvirt configuration (confirmed with a virsh dumpxml).
If you leave the VRs alone after the upgrade, VMs can be provisioned correctly (until you reboot the VRs).

I did also run into the NIO SSL agent not connecting problem again. When I telnetted to 8250, the agent immediately came up without me having to restart it. So keep an eye out for that as well.


- Si

________________________________
From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
Sent: Friday, June 17, 2016 3:02 PM
To: dev@cloudstack.apache.org
Subject: Re: 4.9/master Testing Coordination

Syed is looking into a potential issue with Swift as secondary storage on
master.

ENA is looking into a potential problem when upgrading a VR which was
working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
there are only 2 nics instead of 3.  If they spin a new VR from scratch it
seems to work.  I need to follow up with them to see if they have an
updated status of their testing.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:

> The following tests are running cleanly on KVM with Advanced networking
> and 2 hosts.
>
> echo "Running tests with required_hardware=true"
> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> tags=advanced,required_hardware=true \
> smoke/test_password_server.py \
> smoke/test_vpc_redundant.py \
> smoke/test_routers_iptables_default_policy.py \
> smoke/test_routers_network_ops.py \
> smoke/test_vpc_router_nics.py \
> smoke/test_router_dhcphosts.py \
> smoke/test_loadbalance.py \
> smoke/test_internal_lb.py \
> smoke/test_ssvm.py \
> smoke/test_vpc_vpn.py \
> smoke/test_privategw_acl.py \
> smoke/test_network.py
>
> echo "Running tests with required_hardware=false"
> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> tags=advanced,required_hardware=false \
> smoke/test_routers.py \
> smoke/test_network_acl.py \
> smoke/test_reset_vm_on_reboot.py \
> smoke/test_vm_life_cycle.py \
> smoke/test_service_offerings.py \
> smoke/test_network.py \
> component/test_vpc_offerings.py \
> component/test_vpc_routers.py
>
> I need to do some more manual testing...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> Mike.Tutkowski@netapp.com> wrote:
>
>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>
>> I executed all of the tests in test/integration/plugins/solidfire.
>>
>> They all came back successful.
>> ________________________________________
>> From: John Burwell <jo...@shapeblue.com>
>> Sent: Friday, June 17, 2016 12:56 PM
>> To: dev@cloudstack.apache.org
>> Subject: 4.9/master Testing Coordination
>>
>> All,
>>
>> It is a bit lo-fi, but if you are testing master in preparation for the
>> 4.9 RC, could you please share information about the configurations you
>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>> Any test results could also be helpful.  The hope is to reduce duplication
>> of effort and understand how much of the system has been covered.
>>
>> Thanks,
>> -John
>> john.burwell@shapeblue.com
>> www.shapeblue.com<http://www.shapeblue.com>
ShapeBlue - The CloudStack Company<http://www.shapeblue.com/>
www.shapeblue.com
Overview Apache CloudStack contains an authentication module providing "single sign-on" functionality via the SAML data format. Under certain conditions, a



>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>

Re: 4.9/master Testing Coordination

Posted by Will Stevens <ws...@cloudops.com>.
Syed is looking into a potential issue with Swift as secondary storage on
master.

ENA is looking into a potential problem when upgrading a VR which was
working in 4.8.0, but after an upgrade to 4.9.0 and restarting the network
there are only 2 nics instead of 3.  If they spin a new VR from scratch it
seems to work.  I need to follow up with them to see if they have an
updated status of their testing.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Jun 17, 2016 at 4:00 PM, Will Stevens <ws...@cloudops.com> wrote:

> The following tests are running cleanly on KVM with Advanced networking
> and 2 hosts.
>
> echo "Running tests with required_hardware=true"
> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> tags=advanced,required_hardware=true \
> smoke/test_password_server.py \
> smoke/test_vpc_redundant.py \
> smoke/test_routers_iptables_default_policy.py \
> smoke/test_routers_network_ops.py \
> smoke/test_vpc_router_nics.py \
> smoke/test_router_dhcphosts.py \
> smoke/test_loadbalance.py \
> smoke/test_internal_lb.py \
> smoke/test_ssvm.py \
> smoke/test_vpc_vpn.py \
> smoke/test_privategw_acl.py \
> smoke/test_network.py
>
> echo "Running tests with required_hardware=false"
> nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
> tags=advanced,required_hardware=false \
> smoke/test_routers.py \
> smoke/test_network_acl.py \
> smoke/test_reset_vm_on_reboot.py \
> smoke/test_vm_life_cycle.py \
> smoke/test_service_offerings.py \
> smoke/test_network.py \
> component/test_vpc_offerings.py \
> component/test_vpc_routers.py
>
> I need to do some more manual testing...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <
> Mike.Tutkowski@netapp.com> wrote:
>
>> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>>
>> I executed all of the tests in test/integration/plugins/solidfire.
>>
>> They all came back successful.
>> ________________________________________
>> From: John Burwell <jo...@shapeblue.com>
>> Sent: Friday, June 17, 2016 12:56 PM
>> To: dev@cloudstack.apache.org
>> Subject: 4.9/master Testing Coordination
>>
>> All,
>>
>> It is a bit lo-fi, but if you are testing master in preparation for the
>> 4.9 RC, could you please share information about the configurations you
>> testing (e.g. hypervisors, storage backends, network configurations, etc)?
>> Any test results could also be helpful.  The hope is to reduce duplication
>> of effort and understand how much of the system has been covered.
>>
>> Thanks,
>> -John
>> john.burwell@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>

Re: 4.9/master Testing Coordination

Posted by Will Stevens <ws...@cloudops.com>.
The following tests are running cleanly on KVM with Advanced networking and
2 hosts.

echo "Running tests with required_hardware=true"
nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
tags=advanced,required_hardware=true \
smoke/test_password_server.py \
smoke/test_vpc_redundant.py \
smoke/test_routers_iptables_default_policy.py \
smoke/test_routers_network_ops.py \
smoke/test_vpc_router_nics.py \
smoke/test_router_dhcphosts.py \
smoke/test_loadbalance.py \
smoke/test_internal_lb.py \
smoke/test_ssvm.py \
smoke/test_vpc_vpn.py \
smoke/test_privategw_acl.py \
smoke/test_network.py

echo "Running tests with required_hardware=false"
nosetests --with-marvin --marvin-config=${marvinCfg} -s -a
tags=advanced,required_hardware=false \
smoke/test_routers.py \
smoke/test_network_acl.py \
smoke/test_reset_vm_on_reboot.py \
smoke/test_vm_life_cycle.py \
smoke/test_service_offerings.py \
smoke/test_network.py \
component/test_vpc_offerings.py \
component/test_vpc_routers.py

I need to do some more manual testing...

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Jun 17, 2016 at 3:45 PM, Tutkowski, Mike <Mi...@netapp.com>
wrote:

> My testing has been performed using XenServer 6.5 and ESXi 5.5.
>
> I executed all of the tests in test/integration/plugins/solidfire.
>
> They all came back successful.
> ________________________________________
> From: John Burwell <jo...@shapeblue.com>
> Sent: Friday, June 17, 2016 12:56 PM
> To: dev@cloudstack.apache.org
> Subject: 4.9/master Testing Coordination
>
> All,
>
> It is a bit lo-fi, but if you are testing master in preparation for the
> 4.9 RC, could you please share information about the configurations you
> testing (e.g. hypervisors, storage backends, network configurations, etc)?
> Any test results could also be helpful.  The hope is to reduce duplication
> of effort and understand how much of the system has been covered.
>
> Thanks,
> -John
> john.burwell@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>

Re: 4.9/master Testing Coordination

Posted by "Tutkowski, Mike" <Mi...@netapp.com>.
My testing has been performed using XenServer 6.5 and ESXi 5.5.

I executed all of the tests in test/integration/plugins/solidfire. 

They all came back successful.
________________________________________
From: John Burwell <jo...@shapeblue.com>
Sent: Friday, June 17, 2016 12:56 PM
To: dev@cloudstack.apache.org
Subject: 4.9/master Testing Coordination

All,

It is a bit lo-fi, but if you are testing master in preparation for the 4.9 RC, could you please share information about the configurations you testing (e.g. hypervisors, storage backends, network configurations, etc)?  Any test results could also be helpful.  The hope is to reduce duplication of effort and understand how much of the system has been covered.

Thanks,
-John
john.burwell@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue