You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Raja Pullela <ra...@citrix.com> on 2015/12/15 10:45:07 UTC
BVT report for week of 12/14
Hi,
Here is the BVT report for this week -
- XS Basic - 97.0% (4.6 RC passrate 96.9%)
- XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
- KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
- KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
Summary:
- Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
- test_vpc_vpn.py - couple of tests failing
- test_vpc_redundant.py - couple of test failing
- test_routers_network_ops.py - 5-6 tests failing
- test_routers_iptables_default_policy.py -
- test_privategw_acl.py
Let me know if you have any questions,
Raja
Following are the failed tests:
- XS Basic (both the following are test script issues)
- integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
- KVM Basic (following is a test script issue)
- integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
- XS Adv
- integration.smoke.test_nic.TestNic.test_01_nic
- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_02_isolate_network_FW_PF_default_routes_egress_false
- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
- integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
- integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
- integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
- integration.smoke.test_quota.TestQuota.test_01_quota
- integration.smoke.test_quota.TestQuota.test_02_quota
- integration.smoke.test_quota.TestQuota.test_03_quota
- integration.smoke.test_quota.TestQuota.test_04_quota
- integration.smoke.test_quota.TestQuota.test_05_quota
- integration.smoke.test_quota.TestQuota.test_06_quota
- integration.smoke.test_quota.TestQuota.test_07_quota
- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
- integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_access_vpn
- integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
- integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_haproxy_stats_on_all_interfaces
- integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
- KVM ADV
- integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_vm_from_iso
- integration.smoke.test_routers.TestRouterServices.test_04_restart_network_wo_cleanup
- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
- integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
- integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
- integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
- integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
- integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
- integration.smoke.test_quota.TestQuota.test_01_quota
- integration.smoke.test_quota.TestQuota.test_02_quota
- integration.smoke.test_quota.TestQuota.test_03_quota
- integration.smoke.test_quota.TestQuota.test_04_quota
- integration.smoke.test_quota.TestQuota.test_05_quota
- integration.smoke.test_quota.TestQuota.test_06_quota
- integration.smoke.test_quota.TestQuota.test_07_quota
- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
------------------------------------------------------------------------end-----------------------
RE: BVT report for week of 12/14
Posted by Raja Pullela <ra...@citrix.com>.
Wilder, the failures were due to checkins happened after my report from last week. Let me know if the tests (pl see my last email) you have moved to smoke folder worked in your env. If there are any specific changes/global settings to be done, please let me know,
Best,
Raja
-----Original Message-----
From: Wilder Rodrigues [mailto:WRodrigues@schubergphilis.com]
Sent: Tuesday, December 15, 2015 4:15 PM
To: dev@cloudstack.apache.org
Subject: Re: BVT report for week of 12/14
Raja,
We need your help in order to understand what went wrong. Except for the Quota tests, all the others you mentioned have been executed successfully agains master several times last weeks.
Could you please past the error messages / exception stack trace and also tell us what kind of environment (OS, etc) you have?
Your -1 would be difficult to sustain if all the other people get it to work fine, like Boris, Remi and I already did over the weekend.
Cheers,
Wilder
> On 15 Dec 2015, at 10:45, Raja Pullela <ra...@citrix.com> wrote:
>
> Hi,
>
> Here is the BVT report for this week -
> - XS Basic - 97.0% (4.6 RC passrate 96.9%)
> - XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
> - KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
> - KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to
> 79.7%
>
> Summary:
> - Breakage in Adv Network cases, the following suites need to be run
> both Xen and Adv
> - test_quota.py - all the test failing, we should look at the PR that has checked in this code.
> - test_vpc_vpn.py - couple of tests failing
> - test_vpc_redundant.py - couple of test failing
> - test_routers_network_ops.py - 5-6 tests failing
> - test_routers_iptables_default_policy.py -
> - test_privategw_acl.py
>
> Let me know if you have any questions, Raja
>
> Following are the failed tests:
> - XS Basic (both the following are test script issues)
> -
> integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
> me_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM Basic (following is a test script issue)
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso //known
> testcase issue
>
> - XS Adv
> - integration.smoke.test_nic.TestNic.test_01_nic
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 2_isolate_network_FW_PF_default_routes_egress_false
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> -
> integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
> lesPolicies.test_02_routervm_iptables_policies
> -
> integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
> Policies.test_01_single_VPC_iptables_policies
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 1_isolate_network_FW_PF_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_03_RVR_Network_check_router_state
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
> nt_VPC_default_routes
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_
> redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
> -
> integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
> _routes
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
> vategw_static_routes
> -
> integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_
> access_vpn
> -
> integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
> vpn
> -
> integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_ha
> proxy_stats_on_all_interfaces
> -
> integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
> me_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM ADV
> -
> integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_v
> m_from_iso
> -
> integration.smoke.test_routers.TestRouterServices.test_04_restart_netw
> ork_wo_cleanup
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_03_RVR_Network_check_router_state
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
> redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
> redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> -
> integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
> lesPolicies.test_02_routervm_iptables_policies
> -
> integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
> Policies.test_01_single_VPC_iptables_policies
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 1_isolate_network_FW_PF_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
> nt_VPC_default_routes
> -
> integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
> _routes
> -
> integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_r
> oundrobin_1VPC_3VM_HTTP_port80
> -
> integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
> vpn
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
> vategw_static_routes
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> ----------------------------------------------------------------------
> --end-----------------------
Re: BVT report for week of 12/14
Posted by Wilder Rodrigues <WR...@schubergphilis.com>.
Raja,
We need your help in order to understand what went wrong. Except for the Quota tests, all the others you mentioned have been executed successfully agains master several times last weeks.
Could you please past the error messages / exception stack trace and also tell us what kind of environment (OS, etc) you have?
Your -1 would be difficult to sustain if all the other people get it to work fine, like Boris, Remi and I already did over the weekend.
Cheers,
Wilder
> On 15 Dec 2015, at 10:45, Raja Pullela <ra...@citrix.com> wrote:
>
> Hi,
>
> Here is the BVT report for this week -
> - XS Basic - 97.0% (4.6 RC passrate 96.9%)
> - XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
> - KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
> - KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
>
> Summary:
> - Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
> - test_quota.py - all the test failing, we should look at the PR that has checked in this code.
> - test_vpc_vpn.py - couple of tests failing
> - test_vpc_redundant.py - couple of test failing
> - test_routers_network_ops.py - 5-6 tests failing
> - test_routers_iptables_default_policy.py -
> - test_privategw_acl.py
>
> Let me know if you have any questions,
> Raja
>
> Following are the failed tests:
> - XS Basic (both the following are test script issues)
> - integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM Basic (following is a test script issue)
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
>
> - XS Adv
> - integration.smoke.test_nic.TestNic.test_01_nic
> - integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_02_isolate_network_FW_PF_default_routes_egress_false
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> - integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
> - integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
> - integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
> - integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
> - integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_access_vpn
> - integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
> - integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_haproxy_stats_on_all_interfaces
> - integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM ADV
> - integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_vm_from_iso
> - integration.smoke.test_routers.TestRouterServices.test_04_restart_network_wo_cleanup
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> - integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
> - integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
> - integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
> - integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
> - integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
> - integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> ------------------------------------------------------------------------end-----------------------
RE: BVT report for week of 12/14
Posted by Raja Pullela <ra...@citrix.com>.
Thanks Abhi, was looking at the other PR that checked in the code… the forwarded discuss will help… let me check it out.
From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
Sent: Tuesday, December 15, 2015 5:03 PM
To: dev@cloudstack.apache.org
Subject: Re: BVT report for week of 12/14
Hi Raja,
- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
For testing quota the plugin should be enabled. It is not enabled by default. The discussion is currently going on in this PR:
https://github.com/apache/cloudstack/pull/1240#issuecomment-164652898
-abhi
[ShapeBlue]<http://www.shapeblue.com>
Abhinandan Prateek
Software Architect
,
ShapeBlue
d:
+44 203 603 0540 | s: <tel:+44%20203%20603%200540%20|%20s:>
|
m:
+91 970 11 99011<tel:+91%20970%2011%2099011>
e:
abhinandan.prateek@shapeblue.com | t: <mailto:abhinandan.prateek@shapeblue.com%20|%20t:>
|
w:
www.shapeblue.com<http://www.shapeblue.com>
a:
53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:imageab3322.png@d6adbfad.4388ab0d]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 15/12/15, 3:15 PM, "Raja Pullela" <ra...@citrix.com>> wrote:
>Hi,
>
>Here is the BVT report for this week -
>- XS Basic - 97.0% (4.6 RC passrate 96.9%)
>- XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
>- KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
>- KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
>
>Summary:
>- Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
>- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
>- test_vpc_vpn.py - couple of tests failing
>- test_vpc_redundant.py - couple of test failing
>- test_routers_network_ops.py - 5-6 tests failing
>- test_routers_iptables_default_policy.py -
>- test_privategw_acl.py
>
>Let me know if you have any questions,
>Raja
>
>Following are the failed tests:
>- XS Basic (both the following are test script issues)
>- integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
>- KVM Basic (following is a test script issue)
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
>
>- XS Adv
>- integration.smoke.test_nic.TestNic.test_01_nic
>- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_02_isolate_network_FW_PF_default_routes_egress_false
>- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
>- integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
>- integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
>- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
>- integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
>- integration.smoke.test_quota.TestQuota.test_01_quota
>- integration.smoke.test_quota.TestQuota.test_02_quota
>- integration.smoke.test_quota.TestQuota.test_03_quota
>- integration.smoke.test_quota.TestQuota.test_04_quota
>- integration.smoke.test_quota.TestQuota.test_05_quota
>- integration.smoke.test_quota.TestQuota.test_06_quota
>- integration.smoke.test_quota.TestQuota.test_07_quota
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
>- integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_access_vpn
>- integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
>- integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_haproxy_stats_on_all_interfaces
>- integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
>- KVM ADV
>- integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_vm_from_iso
>- integration.smoke.test_routers.TestRouterServices.test_04_restart_network_wo_cleanup
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
>- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
>- integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
>- integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
>- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
>- integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
>- integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
>- integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
>- integration.smoke.test_quota.TestQuota.test_01_quota
>- integration.smoke.test_quota.TestQuota.test_02_quota
>- integration.smoke.test_quota.TestQuota.test_03_quota
>- integration.smoke.test_quota.TestQuota.test_04_quota
>- integration.smoke.test_quota.TestQuota.test_05_quota
>- integration.smoke.test_quota.TestQuota.test_06_quota
>- integration.smoke.test_quota.TestQuota.test_07_quota
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
>------------------------------------------------------------------------end-----------------------
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build/> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: BVT report for week of 12/14
Posted by Abhinandan Prateek <ab...@shapeblue.com>.
Hi Raja,
- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
For testing quota the plugin should be enabled. It is not enabled by default. The discussion is currently going on in this PR:
https://github.com/apache/cloudstack/pull/1240#issuecomment-164652898
-abhi
[ShapeBlue]<http://www.shapeblue.com>
Abhinandan Prateek
Software Architect , ShapeBlue
d: +44 203 603 0540 | s: <tel:+44%20203%20603%200540%20|%20s:> | m: +91 970 11 99011<tel:+91%20970%2011%2099011>
e: abhinandan.prateek@shapeblue.com | t: <mailto:abhinandan.prateek@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:imageab3322.png@d6adbfad.4388ab0d]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 15/12/15, 3:15 PM, "Raja Pullela" <ra...@citrix.com> wrote:
>Hi,
>
>Here is the BVT report for this week -
>- XS Basic - 97.0% (4.6 RC passrate 96.9%)
>- XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
>- KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
>- KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
>
>Summary:
>- Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
>- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
>- test_vpc_vpn.py - couple of tests failing
>- test_vpc_redundant.py - couple of test failing
>- test_routers_network_ops.py - 5-6 tests failing
>- test_routers_iptables_default_policy.py -
>- test_privategw_acl.py
>
>Let me know if you have any questions,
>Raja
>
>Following are the failed tests:
>- XS Basic (both the following are test script issues)
>- integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
>- KVM Basic (following is a test script issue)
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
>
>- XS Adv
>- integration.smoke.test_nic.TestNic.test_01_nic
>- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_02_isolate_network_FW_PF_default_routes_egress_false
>- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
>- integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
>- integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
>- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
>- integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
>- integration.smoke.test_quota.TestQuota.test_01_quota
>- integration.smoke.test_quota.TestQuota.test_02_quota
>- integration.smoke.test_quota.TestQuota.test_03_quota
>- integration.smoke.test_quota.TestQuota.test_04_quota
>- integration.smoke.test_quota.TestQuota.test_05_quota
>- integration.smoke.test_quota.TestQuota.test_06_quota
>- integration.smoke.test_quota.TestQuota.test_07_quota
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
>- integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_access_vpn
>- integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
>- integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_haproxy_stats_on_all_interfaces
>- integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
>- KVM ADV
>- integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_vm_from_iso
>- integration.smoke.test_routers.TestRouterServices.test_04_restart_network_wo_cleanup
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
>- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
>- integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
>- integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
>- integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
>- integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
>- integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
>- integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
>- integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
>- integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
>- integration.smoke.test_quota.TestQuota.test_01_quota
>- integration.smoke.test_quota.TestQuota.test_02_quota
>- integration.smoke.test_quota.TestQuota.test_03_quota
>- integration.smoke.test_quota.TestQuota.test_04_quota
>- integration.smoke.test_quota.TestQuota.test_05_quota
>- integration.smoke.test_quota.TestQuota.test_06_quota
>- integration.smoke.test_quota.TestQuota.test_07_quota
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
>- integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
>- integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
>------------------------------------------------------------------------end-----------------------
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: BVT report for week of 12/14
Posted by Wilder Rodrigues <WR...@schubergphilis.com>.
Super!
Thanks, man!
Cheers,
Wilder
> On 15 Dec 2015, at 12:57, Raja Pullela <ra...@citrix.com> wrote:
>
> Ok, will cross check the configurations and rerun them.
>
> -----Original Message-----
> From: Wilder Rodrigues [mailto:WRodrigues@schubergphilis.com]
> Sent: Tuesday, December 15, 2015 5:22 PM
> To: dev@cloudstack.apache.org
> Subject: Re: BVT report for week of 12/14
>
> Hi Raja,
>
> Yes, those were moved and have been tested - as they to smoke tests instead of component. All of them are passing and they have been executed not only by me. There are all green on both KVM and Xen.
>
> You can find my DC configurations here:
>
> https://github.com/schubergphilis/MCT-shared/blob/master/marvin/mct-zone1-kvm1-ISOLATED.cfg
> https://github.com/schubergphilis/MCT-shared/blob/master/marvin/mct-zone1-xen1-ISOLATED.cfg
>
> But the tests do not depend on anything special.
>
> The templates I use can be found here: http://dl.openvm.eu/cloudstack/macchinina/x86_64/
>
> Cheers,
> Wilder
>
> On 15 Dec 2015, at 12:37, Raja Pullela <ra...@citrix.com>> wrote:
>
> Sebastien, Yes, was looking at the details, found out the following details on the failures:
>
> 1) CLOUDSTACK-9120 - Moves tests to the smoke directory - test_router_dhcphosts.py, test_routers_iptables_default_policy.py, test_routers_network_ops.py, test_vpc_redundant.py, test_vpc_router_nics.py. the changes were done around a week ago.
>
> 2) CLOUDSTACK-9153: When negative credits are added to an account- test_quota.py. This one added by Abhinandan Prateek. I did see the PR related this with results showing it is passing. However, in my env these tests were failing with error " global name 'quotaTariffList' is not defined".
>
> Wilder, can you comment on if all the tests in these scripts were passing in your env?
>
> Best,
> Raja
> -----Original Message-----
> From: Sebastien Goasguen [mailto:runseb@gmail.com]
> Sent: Tuesday, December 15, 2015 3:31 PM
> To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
> Subject: Re: BVT report for week of 12/14
>
>
> On Dec 15, 2015, at 10:45 AM, Raja Pullela <ra...@citrix.com>> wrote:
>
> Hi,
>
> Here is the BVT report for this week -
> - XS Basic - 97.0% (4.6 RC passrate 96.9%)
> - XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
>
> Hi Raja, with very few commits getting into master it should be relatively easy to identify the commits responsible for this.
> Do you know which one(s) it was.
>
> - KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
> - KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
>
> Summary:
> - Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
> - test_quota.py - all the test failing, we should look at the PR that has checked in this code.
> - test_vpc_vpn.py - couple of tests failing
> - test_vpc_redundant.py - couple of test failing
> - test_routers_network_ops.py - 5-6 tests failing
> - test_routers_iptables_default_policy.py -
> - test_privategw_acl.py
>
> Let me know if you have any questions, Raja
>
> Following are the failed tests:
> - XS Basic (both the following are test script issues)
> -
> integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
> me_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM Basic (following is a test script issue)
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
>
> - XS Adv
> - integration.smoke.test_nic.TestNic.test_01_nic
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 2_isolate_network_FW_PF_default_routes_egress_false
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> -
> integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
> lesPolicies.test_02_routervm_iptables_policies
> -
> integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
> Policies.test_01_single_VPC_iptables_policies
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 1_isolate_network_FW_PF_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_03_RVR_Network_check_router_state
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
> nt_VPC_default_routes
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_
> redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
> -
> integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
> _routes
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
> vategw_static_routes
> -
> integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_
> access_vpn
> -
> integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
> vpn
> -
> integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_ha
> proxy_stats_on_all_interfaces
> -
> integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
> me_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM ADV
> -
> integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_v
> m_from_iso
> -
> integration.smoke.test_routers.TestRouterServices.test_04_restart_netw
> ork_wo_cleanup
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_03_RVR_Network_check_router_state
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
> redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
> redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> -
> integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
> lesPolicies.test_02_routervm_iptables_policies
> -
> integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
> Policies.test_01_single_VPC_iptables_policies
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 1_isolate_network_FW_PF_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
> nt_VPC_default_routes
> -
> integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
> _routes
> -
> integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_r
> oundrobin_1VPC_3VM_HTTP_port80
> -
> integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
> vpn
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
> vategw_static_routes
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> ----------------------------------------------------------------------
> --end-----------------------
>
>
RE: BVT report for week of 12/14
Posted by Raja Pullela <ra...@citrix.com>.
Ok, will cross check the configurations and rerun them.
-----Original Message-----
From: Wilder Rodrigues [mailto:WRodrigues@schubergphilis.com]
Sent: Tuesday, December 15, 2015 5:22 PM
To: dev@cloudstack.apache.org
Subject: Re: BVT report for week of 12/14
Hi Raja,
Yes, those were moved and have been tested - as they to smoke tests instead of component. All of them are passing and they have been executed not only by me. There are all green on both KVM and Xen.
You can find my DC configurations here:
https://github.com/schubergphilis/MCT-shared/blob/master/marvin/mct-zone1-kvm1-ISOLATED.cfg
https://github.com/schubergphilis/MCT-shared/blob/master/marvin/mct-zone1-xen1-ISOLATED.cfg
But the tests do not depend on anything special.
The templates I use can be found here: http://dl.openvm.eu/cloudstack/macchinina/x86_64/
Cheers,
Wilder
On 15 Dec 2015, at 12:37, Raja Pullela <ra...@citrix.com>> wrote:
Sebastien, Yes, was looking at the details, found out the following details on the failures:
1) CLOUDSTACK-9120 - Moves tests to the smoke directory - test_router_dhcphosts.py, test_routers_iptables_default_policy.py, test_routers_network_ops.py, test_vpc_redundant.py, test_vpc_router_nics.py. the changes were done around a week ago.
2) CLOUDSTACK-9153: When negative credits are added to an account- test_quota.py. This one added by Abhinandan Prateek. I did see the PR related this with results showing it is passing. However, in my env these tests were failing with error " global name 'quotaTariffList' is not defined".
Wilder, can you comment on if all the tests in these scripts were passing in your env?
Best,
Raja
-----Original Message-----
From: Sebastien Goasguen [mailto:runseb@gmail.com]
Sent: Tuesday, December 15, 2015 3:31 PM
To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
Subject: Re: BVT report for week of 12/14
On Dec 15, 2015, at 10:45 AM, Raja Pullela <ra...@citrix.com>> wrote:
Hi,
Here is the BVT report for this week -
- XS Basic - 97.0% (4.6 RC passrate 96.9%)
- XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
Hi Raja, with very few commits getting into master it should be relatively easy to identify the commits responsible for this.
Do you know which one(s) it was.
- KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
- KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
Summary:
- Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
- test_vpc_vpn.py - couple of tests failing
- test_vpc_redundant.py - couple of test failing
- test_routers_network_ops.py - 5-6 tests failing
- test_routers_iptables_default_policy.py -
- test_privategw_acl.py
Let me know if you have any questions, Raja
Following are the failed tests:
- XS Basic (both the following are test script issues)
-
integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
me_snapshot
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
- KVM Basic (following is a test script issue)
- integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
- XS Adv
- integration.smoke.test_nic.TestNic.test_01_nic
-
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
2_isolate_network_FW_PF_default_routes_egress_false
- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
-
integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
lesPolicies.test_02_routervm_iptables_policies
-
integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
Policies.test_01_single_VPC_iptables_policies
-
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
1_isolate_network_FW_PF_default_routes_egress_true
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_03_RVR_Network_check_router_state
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
nt_VPC_default_routes
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_
redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
-
integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
_routes
- integration.smoke.test_quota.TestQuota.test_01_quota
- integration.smoke.test_quota.TestQuota.test_02_quota
- integration.smoke.test_quota.TestQuota.test_03_quota
- integration.smoke.test_quota.TestQuota.test_04_quota
- integration.smoke.test_quota.TestQuota.test_05_quota
- integration.smoke.test_quota.TestQuota.test_06_quota
- integration.smoke.test_quota.TestQuota.test_07_quota
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
vategw_static_routes
-
integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_
access_vpn
-
integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
vpn
-
integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_ha
proxy_stats_on_all_interfaces
-
integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
me_snapshot
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
- KVM ADV
-
integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_v
m_from_iso
-
integration.smoke.test_routers.TestRouterServices.test_04_restart_netw
ork_wo_cleanup
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_03_RVR_Network_check_router_state
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
-
integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
lesPolicies.test_02_routervm_iptables_policies
-
integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
Policies.test_01_single_VPC_iptables_policies
-
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
1_isolate_network_FW_PF_default_routes_egress_true
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
nt_VPC_default_routes
-
integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
_routes
-
integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_r
oundrobin_1VPC_3VM_HTTP_port80
-
integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
vpn
- integration.smoke.test_quota.TestQuota.test_01_quota
- integration.smoke.test_quota.TestQuota.test_02_quota
- integration.smoke.test_quota.TestQuota.test_03_quota
- integration.smoke.test_quota.TestQuota.test_04_quota
- integration.smoke.test_quota.TestQuota.test_05_quota
- integration.smoke.test_quota.TestQuota.test_06_quota
- integration.smoke.test_quota.TestQuota.test_07_quota
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
vategw_static_routes
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
----------------------------------------------------------------------
--end-----------------------
Re: BVT report for week of 12/14
Posted by Wilder Rodrigues <WR...@schubergphilis.com>.
Hi Raja,
Yes, those were moved and have been tested - as they to smoke tests instead of component. All of them are passing and they have been executed not only by me. There are all green on both KVM and Xen.
You can find my DC configurations here:
https://github.com/schubergphilis/MCT-shared/blob/master/marvin/mct-zone1-kvm1-ISOLATED.cfg
https://github.com/schubergphilis/MCT-shared/blob/master/marvin/mct-zone1-xen1-ISOLATED.cfg
But the tests do not depend on anything special.
The templates I use can be found here: http://dl.openvm.eu/cloudstack/macchinina/x86_64/
Cheers,
Wilder
On 15 Dec 2015, at 12:37, Raja Pullela <ra...@citrix.com>> wrote:
Sebastien, Yes, was looking at the details, found out the following details on the failures:
1) CLOUDSTACK-9120 - Moves tests to the smoke directory - test_router_dhcphosts.py, test_routers_iptables_default_policy.py, test_routers_network_ops.py, test_vpc_redundant.py, test_vpc_router_nics.py. the changes were done around a week ago.
2) CLOUDSTACK-9153: When negative credits are added to an account- test_quota.py. This one added by Abhinandan Prateek. I did see the PR related this with results showing it is passing. However, in my env these tests were failing with error " global name 'quotaTariffList' is not defined".
Wilder, can you comment on if all the tests in these scripts were passing in your env?
Best,
Raja
-----Original Message-----
From: Sebastien Goasguen [mailto:runseb@gmail.com]
Sent: Tuesday, December 15, 2015 3:31 PM
To: dev@cloudstack.apache.org<ma...@cloudstack.apache.org>
Subject: Re: BVT report for week of 12/14
On Dec 15, 2015, at 10:45 AM, Raja Pullela <ra...@citrix.com>> wrote:
Hi,
Here is the BVT report for this week -
- XS Basic - 97.0% (4.6 RC passrate 96.9%)
- XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
Hi Raja, with very few commits getting into master it should be relatively easy to identify the commits responsible for this.
Do you know which one(s) it was.
- KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
- KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to
79.7%
Summary:
- Breakage in Adv Network cases, the following suites need to be run
both Xen and Adv
- test_quota.py - all the test failing, we should look at the PR that has checked in this code.
- test_vpc_vpn.py - couple of tests failing
- test_vpc_redundant.py - couple of test failing
- test_routers_network_ops.py - 5-6 tests failing
- test_routers_iptables_default_policy.py -
- test_privategw_acl.py
Let me know if you have any questions, Raja
Following are the failed tests:
- XS Basic (both the following are test script issues)
-
integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
me_snapshot
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
- KVM Basic (following is a test script issue)
- integration.smoke.test_iso.TestISO.test_07_list_default_iso //known
testcase issue
- XS Adv
- integration.smoke.test_nic.TestNic.test_01_nic
-
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
2_isolate_network_FW_PF_default_routes_egress_false
- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
-
integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
lesPolicies.test_02_routervm_iptables_policies
-
integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
Policies.test_01_single_VPC_iptables_policies
-
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
1_isolate_network_FW_PF_default_routes_egress_true
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_03_RVR_Network_check_router_state
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
nt_VPC_default_routes
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_
redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
-
integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
_routes
- integration.smoke.test_quota.TestQuota.test_01_quota
- integration.smoke.test_quota.TestQuota.test_02_quota
- integration.smoke.test_quota.TestQuota.test_03_quota
- integration.smoke.test_quota.TestQuota.test_04_quota
- integration.smoke.test_quota.TestQuota.test_05_quota
- integration.smoke.test_quota.TestQuota.test_06_quota
- integration.smoke.test_quota.TestQuota.test_07_quota
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
vategw_static_routes
-
integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_
access_vpn
-
integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
vpn
-
integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_ha
proxy_stats_on_all_interfaces
-
integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
me_snapshot
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
- KVM ADV
-
integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_v
m_from_iso
-
integration.smoke.test_routers.TestRouterServices.test_04_restart_netw
ork_wo_cleanup
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_03_RVR_Network_check_router_state
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
- <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
-
integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
lesPolicies.test_02_routervm_iptables_policies
-
integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
Policies.test_01_single_VPC_iptables_policies
-
integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
1_isolate_network_FW_PF_default_routes_egress_true
-
integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
-
integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
nt_VPC_default_routes
-
integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
_routes
-
integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_r
oundrobin_1VPC_3VM_HTTP_port80
-
integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
vpn
- integration.smoke.test_quota.TestQuota.test_01_quota
- integration.smoke.test_quota.TestQuota.test_02_quota
- integration.smoke.test_quota.TestQuota.test_03_quota
- integration.smoke.test_quota.TestQuota.test_04_quota
- integration.smoke.test_quota.TestQuota.test_05_quota
- integration.smoke.test_quota.TestQuota.test_06_quota
- integration.smoke.test_quota.TestQuota.test_07_quota
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
ategw_static_routes
-
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
vategw_static_routes
- integration.smoke.test_iso.TestISO.test_07_list_default_iso
----------------------------------------------------------------------
--end-----------------------
RE: BVT report for week of 12/14
Posted by Raja Pullela <ra...@citrix.com>.
Sebastien, Yes, was looking at the details, found out the following details on the failures:
1) CLOUDSTACK-9120 - Moves tests to the smoke directory - test_router_dhcphosts.py, test_routers_iptables_default_policy.py, test_routers_network_ops.py, test_vpc_redundant.py, test_vpc_router_nics.py. the changes were done around a week ago.
2) CLOUDSTACK-9153: When negative credits are added to an account- test_quota.py. This one added by Abhinandan Prateek. I did see the PR related this with results showing it is passing. However, in my env these tests were failing with error " global name 'quotaTariffList' is not defined".
Wilder, can you comment on if all the tests in these scripts were passing in your env?
Best,
Raja
-----Original Message-----
From: Sebastien Goasguen [mailto:runseb@gmail.com]
Sent: Tuesday, December 15, 2015 3:31 PM
To: dev@cloudstack.apache.org
Subject: Re: BVT report for week of 12/14
> On Dec 15, 2015, at 10:45 AM, Raja Pullela <ra...@citrix.com> wrote:
>
> Hi,
>
> Here is the BVT report for this week -
> - XS Basic - 97.0% (4.6 RC passrate 96.9%)
> - XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
Hi Raja, with very few commits getting into master it should be relatively easy to identify the commits responsible for this.
Do you know which one(s) it was.
> - KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
> - KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to
> 79.7%
>
> Summary:
> - Breakage in Adv Network cases, the following suites need to be run
> both Xen and Adv
> - test_quota.py - all the test failing, we should look at the PR that has checked in this code.
> - test_vpc_vpn.py - couple of tests failing
> - test_vpc_redundant.py - couple of test failing
> - test_routers_network_ops.py - 5-6 tests failing
> - test_routers_iptables_default_policy.py -
> - test_privategw_acl.py
>
> Let me know if you have any questions, Raja
>
> Following are the failed tests:
> - XS Basic (both the following are test script issues)
> -
> integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
> me_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM Basic (following is a test script issue)
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso //known
> testcase issue
>
> - XS Adv
> - integration.smoke.test_nic.TestNic.test_01_nic
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 2_isolate_network_FW_PF_default_routes_egress_false
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> -
> integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
> lesPolicies.test_02_routervm_iptables_policies
> -
> integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
> Policies.test_01_single_VPC_iptables_policies
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 1_isolate_network_FW_PF_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_03_RVR_Network_check_router_state
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
> nt_VPC_default_routes
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_
> redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
> -
> integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
> _routes
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
> vategw_static_routes
> -
> integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_
> access_vpn
> -
> integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
> vpn
> -
> integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_ha
> proxy_stats_on_all_interfaces
> -
> integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volu
> me_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM ADV
> -
> integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_v
> m_from_iso
> -
> integration.smoke.test_routers.TestRouterServices.test_04_restart_netw
> ork_wo_cleanup
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_03_RVR_Network_check_router_state
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
> redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_
> redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> -
> integration.smoke.test_routers_iptables_default_policy.TestRouterIpTab
> lesPolicies.test_02_routervm_iptables_policies
> -
> integration.smoke.test_routers_iptables_default_policy.TestVPCIpTables
> Policies.test_01_single_VPC_iptables_policies
> -
> integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_0
> 1_isolate_network_FW_PF_default_routes_egress_true
> -
> integration.smoke.test_routers_network_ops.TestRedundantIsolateNetwork
> s.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> -
> integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redunda
> nt_VPC_default_routes
> -
> integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default
> _routes
> -
> integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_r
> oundrobin_1VPC_3VM_HTTP_port80
> -
> integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_
> vpn
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_priv
> ategw_static_routes
> -
> integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_pri
> vategw_static_routes
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> ----------------------------------------------------------------------
> --end-----------------------
Re: BVT report for week of 12/14
Posted by Sebastien Goasguen <ru...@gmail.com>.
> On Dec 15, 2015, at 10:45 AM, Raja Pullela <ra...@citrix.com> wrote:
>
> Hi,
>
> Here is the BVT report for this week -
> - XS Basic - 97.0% (4.6 RC passrate 96.9%)
> - XS Adv - 79.5% (4.6 RC passrate 93.4%) // went down from 92.7% to 79.5%
Hi Raja, with very few commits getting into master it should be relatively easy to identify the commits responsible for this.
Do you know which one(s) it was.
> - KVM Basic - 98.6% (4.6 RC passrate - 98.6%)
> - KVM Adv - 79.7% (4.6 RC passrate - 96.3%) // went down from 96.4% to 79.7%
>
> Summary:
> - Breakage in Adv Network cases, the following suites need to be run both Xen and Adv
> - test_quota.py - all the test failing, we should look at the PR that has checked in this code.
> - test_vpc_vpn.py - couple of tests failing
> - test_vpc_redundant.py - couple of test failing
> - test_routers_network_ops.py - 5-6 tests failing
> - test_routers_iptables_default_policy.py -
> - test_privategw_acl.py
>
> Let me know if you have any questions,
> Raja
>
> Following are the failed tests:
> - XS Basic (both the following are test script issues)
> - integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM Basic (following is a test script issue)
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso //known testcase issue
>
> - XS Adv
> - integration.smoke.test_nic.TestNic.test_01_nic
> - integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_02_isolate_network_FW_PF_default_routes_egress_false
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> - integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
> - integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
> - integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
> - integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
> - integration.smoke.test_vpc_vpn.TestVpcRemoteAccessVpn.test_vpc_remote_access_vpn
> - integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
> - integration.smoke.test_internal_lb.TestInternalLb.test02_internallb_haproxy_stats_on_all_interfaces
> - integration.smoke.test_vm_snapshots.TestSnapshots.test_01_test_vm_volume_snapshot
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> - KVM ADV
> - integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO.test_deploy_vm_from_iso
> - integration.smoke.test_routers.TestRouterServices.test_04_restart_network_wo_cleanup
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_03_RVR_Network_check_router_state
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
> - <nose.suite.ContextSuite context=TestRouterDHCPHosts>:setup
> - integration.smoke.test_routers_iptables_default_policy.TestRouterIpTablesPolicies.test_02_routervm_iptables_policies
> - integration.smoke.test_routers_iptables_default_policy.TestVPCIpTablesPolicies.test_01_single_VPC_iptables_policies
> - integration.smoke.test_routers_network_ops.TestIsolatedNetworks.test_01_isolate_network_FW_PF_default_routes_egress_true
> - integration.smoke.test_routers_network_ops.TestRedundantIsolateNetworks.test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
> - integration.smoke.test_vpc_redundant.TestVPCRedundancy.test_02_redundant_VPC_default_routes
> - integration.smoke.test_vpc_router_nics.TestVPCNics.test_02_VPC_default_routes
> - integration.smoke.test_internal_lb.TestInternalLb.test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
> - integration.smoke.test_vpc_vpn.TestVpcSite2SiteVpn.test_vpc_site2site_vpn
> - integration.smoke.test_quota.TestQuota.test_01_quota
> - integration.smoke.test_quota.TestQuota.test_02_quota
> - integration.smoke.test_quota.TestQuota.test_03_quota
> - integration.smoke.test_quota.TestQuota.test_04_quota
> - integration.smoke.test_quota.TestQuota.test_05_quota
> - integration.smoke.test_quota.TestQuota.test_06_quota
> - integration.smoke.test_quota.TestQuota.test_07_quota
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_02_vpc_privategw_static_routes
> - integration.smoke.test_privategw_acl.TestPrivateGwACL.test_03_rvpc_privategw_static_routes
> - integration.smoke.test_iso.TestISO.test_07_list_default_iso
>
> ------------------------------------------------------------------------end-----------------------