You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Rayees Namathponnan <ra...@citrix.com> on 2014/01/21 06:43:33 UTC
4.3 Regression automation result - Xen 6.2
Hi All,
Here latest regression automation result with 4.3 build on Xen 6.2 ; Total pass rate is 93.69 %
Automation executed from http://jenkins.buildacloud.org
Regression Tests
===========
On Xenserver 6.2 / Test Run: #285 and #286 (re run)
--------
Total:444
Fail :28
Skip :22
--------
Job:
http://jenkins.buildacloud.org/view/cloudstack-qa-4.3/job/test-regression-matrix/285/
http://jenkins.buildacloud.org/view/cloudstack-qa-4.3/job/test-regression-matrix/286
Logs
http://jenkins.buildacloud.org/job/test-matrix-extended/distro=centos63,hypervisor=xen,profile=xen62/254/artifact/254.tar.bz2
Configuration Name All Failed Skipped
suite=test_security_groups 1 0 0
suite=test_multiple_ip_ranges 1 0 0
suite=test_vm_passwdenabled 1 0 0
suite=test_project_configs 5 0 3
suite=test_project_usage 9 0 0
suite=test_volumes 9 0 0
suite=test_snapshot_limits 1 0 0
suite=test_templates 4 0 0
suite=test_eip_elb 1 0 0
suite=test_project_limits 8 1 0
suite=test_usage 9 0 0
suite=test_stopped_vm 17 1 0
suite=test_vpc_routers 10 2 0
suite=test_affinity_groups 42 0 1
suite=test_routers 6 0 0
suite=test_storage_motion 2 0 1
suite=test_egress_rules 1 0 0
suite=test_blocker_bugs 7 0 0
suite=test_vpc_network 27 3 9
suite=test_vpc_vms_deployment 7 1 0
suite=test_network_offering 3 0 0
suite=test_vpc 20 1 0
suite=test_egress_fw_rules 26 2 4
suite=test_haproxy 11 1 0
suite=test_allocation_states 6 0 0
suite=test_assign_vm 16 0 0
suite=test_vpc_network_lbrules 15 2 0
suite=test_redundant_router_cleanups 3 0 0
suite=test_update_vm 1 0 0
suite=test_vpn_users 7 0 0
suite=test_reset_ssh_keypair 10 4 0
suite=test_implicit_planner 1 0 1
suite=test_vpc_offerings 8 0 0
suite=test_redundant_router_services 1 0 0
suite=test_recurring_snapshots 2 0 0
suite=test_project_resources 7 1 0
suite=test_regions 3 0 0
suite=test_snapshot_gc 1 0 0
suite=test_vpc_vm_life_cycle 33 4 0
suite=test_redundant_router_upgrades 2 0 0
suite=test_explicit_dedication 1 0 1
suite=test_snapshots 7 0 0
suite=test_projects 10 0 0
suite=test_vpc_network_staticnatrule 6 2 0
suite=test_accounts 14 1 0
suite=test_advancedsg_networks 1 0 0
suite=test_ip_reservation 4 0 0
suite=test_tags 20 0 1
suite=test_shared_networks 13 0 1
suite=test_vpc_network_pfrules 10 2 0
suite=test_resource_limits 11 0 0
suite=test_regions_accounts 3 0 0
Total : 444
Failed : 28
Skip : 22
Most the failures are due to timeout error
Regards,
Rayees
Re: 4.3 Regression automation result - Xen 6.2
Posted by Daan Hoogland <da...@gmail.com>.
Thanks Rayees, great!!!
On Tue, Jan 21, 2014 at 4:51 PM, Rayees Namathponnan
<ra...@citrix.com> wrote:
> Timeout failures are intermediate, If we re-run these test individually
> test gets pass.
>
> I observed similar issues in previous run also, in my analysis I didn¹t
> see any product failures with this run.
>
> Regards,
> Rayees
>
>
> On 1/21/14 7:28 AM, "Daan Hoogland" <da...@gmail.com> wrote:
>
>>On Tue, Jan 21, 2014 at 6:43 AM, Rayees Namathponnan
>><ra...@citrix.com> wrote:
>>> Total : 444
>>> Failed : 28
>>> Skip : 22
>>>
>>> Most the failures are due to timeout error
>>>
>>> Regards,
>>
>>
>>Rayees, one litle query. Are those timeout issues intermittend over
>>runs? Meaning do different tests fail each run? Do you have data on
>>that? If so I think we can ignore them but if not we have a problem.
>>We then cannot assess how serious they are. Any thoughts?
>>
>>Daan
>
Re: 4.3 Regression automation result - Xen 6.2
Posted by Rayees Namathponnan <ra...@citrix.com>.
Timeout failures are intermediate, If we re-run these test individually
test gets pass.
I observed similar issues in previous run also, in my analysis I didn¹t
see any product failures with this run.
Regards,
Rayees
On 1/21/14 7:28 AM, "Daan Hoogland" <da...@gmail.com> wrote:
>On Tue, Jan 21, 2014 at 6:43 AM, Rayees Namathponnan
><ra...@citrix.com> wrote:
>> Total : 444
>> Failed : 28
>> Skip : 22
>>
>> Most the failures are due to timeout error
>>
>> Regards,
>
>
>Rayees, one litle query. Are those timeout issues intermittend over
>runs? Meaning do different tests fail each run? Do you have data on
>that? If so I think we can ignore them but if not we have a problem.
>We then cannot assess how serious they are. Any thoughts?
>
>Daan
Re: 4.3 Regression automation result - Xen 6.2
Posted by Daan Hoogland <da...@gmail.com>.
On Tue, Jan 21, 2014 at 6:43 AM, Rayees Namathponnan
<ra...@citrix.com> wrote:
> Total : 444
> Failed : 28
> Skip : 22
>
> Most the failures are due to timeout error
>
> Regards,
Rayees, one litle query. Are those timeout issues intermittend over
runs? Meaning do different tests fail each run? Do you have data on
that? If so I think we can ignore them but if not we have a problem.
We then cannot assess how serious they are. Any thoughts?
Daan