You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Raja Pullela <ra...@citrix.com> on 2015/09/24 21:08:03 UTC
BVT report 9/23
BVT report 09/23
simulator basic - 30% , earlier runs had 100% pass rate, failures need to be analyzed
Simulator adv - 50%, earlier runs had 100% pass rate, failures need to be analyzed
XS basic - 95.3%
XS Adv - 93.6%
XS eip - 86.7%
KVM basic - 89.4%
KVM Adv - Deployment issue, need to check and will update
KVM eip - 95.7%
VMware Adv - deployment issue, need to check and update
RE: BVT report 9/23
Posted by Raja Pullela <ra...@citrix.com>.
Yes Remi... I will look into the failures and report.
-----Original Message-----
From: Remi Bergsma [mailto:RBergsma@schubergphilis.com]
Sent: Friday, September 25, 2015 12:48 PM
To: dev@cloudstack.apache.org
Subject: Re: BVT report 9/23
Hi Raja,
Thanks for the report. Can you please add details, so we can see what tests exactly fail? We can then try to reproduce / root cause.
Regards, Remi
> On 24 Sep 2015, at 21:09, Raja Pullela <ra...@citrix.com> wrote:
>
> BVT report 09/23
>
> simulator basic - 30% , earlier runs had 100% pass rate, failures need
> to be analyzed Simulator adv - 50%, earlier runs had 100% pass rate,
> failures need to be analyzed XS basic - 95.3% XS Adv - 93.6% XS eip -
> 86.7% KVM basic - 89.4% KVM Adv - Deployment issue, need to check and
> will update KVM eip - 95.7% VMware Adv - deployment issue, need to
> check and update
>
>
Re: BVT report 9/23
Posted by Remi Bergsma <RB...@schubergphilis.com>.
Hi Raja,
Thanks for the report. Can you please add details, so we can see what tests exactly fail? We can then try to reproduce / root cause.
Regards, Remi
> On 24 Sep 2015, at 21:09, Raja Pullela <ra...@citrix.com> wrote:
>
> BVT report 09/23
>
> simulator basic - 30% , earlier runs had 100% pass rate, failures need to be analyzed
> Simulator adv - 50%, earlier runs had 100% pass rate, failures need to be analyzed
> XS basic - 95.3%
> XS Adv - 93.6%
> XS eip - 86.7%
> KVM basic - 89.4%
> KVM Adv - Deployment issue, need to check and will update
> KVM eip - 95.7%
> VMware Adv - deployment issue, need to check and update
>
>
Re: BVT report 9/23
Posted by Sebastien Goasguen <ru...@gmail.com>.
> On Sep 25, 2015, at 3:25 AM, Raja Pullela <ra...@citrix.com> wrote:
>
> Please see inline comments..
>
>> On Sep 25, 2015, at 1:47 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>>
>>
>>> On Sep 24, 2015, at 9:08 PM, Raja Pullela <ra...@citrix.com> wrote:
>>>
>>> BVT report 09/23
>>>
>>> simulator basic - 30% , earlier runs had 100% pass rate, failures need to be analyzed
>>
>> What’s earlier ? yesterday, last week, six months ago ?
> Earlier - yesterday's runs
Ok so it is much wore than yesterday. I look forward to get your analysis .
>>
>>> Simulator adv - 50%, earlier runs had 100% pass rate, failures need to be analyzed
>>
>> Same here, what’s earlier ?
> Earlier - yesterday's runs
>>
>>> XS basic - 95.3%
>>> XS Adv - 93.6%
>>> XS eip - 86.7%
>>
>> Is this good enough from your view for release, or do you want 100% ?
> We should get 100%
>>
>>> KVM basic - 89.4%
>>> KVM Adv - Deployment issue, need to check and will update
>>> KVM eip - 95.7%
>>
>> Good so it’s running now.
> Yes, after the agent fixes this is back up
Was this fix identified as a blocker, was there a PR for it and has it been merged ?
The reason I am asking is that it is difficult to understand these numbers out of context.
Seeing the results given some notion of the success, but we don’t really no what tests are run, on what exact setup.
Also you mentioned regression tests..what are those ? we don’t run regression tests upstream, so I suppose these are internal Citrix tests...
>>
>>> VMware Adv - deployment issue, need to check and update
>>
Re: BVT report 9/23
Posted by Sebastien Goasguen <ru...@gmail.com>.
> On Sep 25, 2015, at 8:14 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
>
>> On Sep 25, 2015, at 3:25 AM, Raja Pullela <ra...@citrix.com> wrote:
>>
>> Please see inline comments..
>>
>>> On Sep 25, 2015, at 1:47 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>>>
>>>
>>>> On Sep 24, 2015, at 9:08 PM, Raja Pullela <ra...@citrix.com> wrote:
>>>>
>>>> BVT report 09/23
>>>>
>>>> simulator basic - 30% , earlier runs had 100% pass rate, failures need to be analyzed
>>>
>>> What’s earlier ? yesterday, last week, six months ago ?
>> Earlier - yesterday's runs
>
> Ok so it is much wore than yesterday. I look forward to get your analysis .
Raja, to try to understand this I just looked at the commits on master.
Yesterday there was one commit, related to Vagrantfiles for devcloud.
There is no chance that this commit can affect master and the tests…so this must be a glitch in the simulator runs.
-sebastien
>
>>>
>>>> Simulator adv - 50%, earlier runs had 100% pass rate, failures need to be analyzed
>>>
>>> Same here, what’s earlier ?
>> Earlier - yesterday's runs
>>>
>>>> XS basic - 95.3%
>>>> XS Adv - 93.6%
>>>> XS eip - 86.7%
>>>
>>> Is this good enough from your view for release, or do you want 100% ?
>> We should get 100%
>>>
>>>> KVM basic - 89.4%
>>>> KVM Adv - Deployment issue, need to check and will update
>>>> KVM eip - 95.7%
>>>
>>> Good so it’s running now.
>> Yes, after the agent fixes this is back up
>
> Was this fix identified as a blocker, was there a PR for it and has it been merged ?
>
>
> The reason I am asking is that it is difficult to understand these numbers out of context.
> Seeing the results given some notion of the success, but we don’t really no what tests are run, on what exact setup.
>
> Also you mentioned regression tests..what are those ? we don’t run regression tests upstream, so I suppose these are internal Citrix tests...
>
>
>>>
>>>> VMware Adv - deployment issue, need to check and update
>>>
>
Re: BVT report 9/23
Posted by Raja Pullela <ra...@citrix.com>.
Please see inline comments..
> On Sep 25, 2015, at 1:47 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
>
>> On Sep 24, 2015, at 9:08 PM, Raja Pullela <ra...@citrix.com> wrote:
>>
>> BVT report 09/23
>>
>> simulator basic - 30% , earlier runs had 100% pass rate, failures need to be analyzed
>
> What’s earlier ? yesterday, last week, six months ago ?
Earlier - yesterday's runs
>
>> Simulator adv - 50%, earlier runs had 100% pass rate, failures need to be analyzed
>
> Same here, what’s earlier ?
Earlier - yesterday's runs
>
>> XS basic - 95.3%
>> XS Adv - 93.6%
>> XS eip - 86.7%
>
> Is this good enough from your view for release, or do you want 100% ?
We should get 100%
>
>> KVM basic - 89.4%
>> KVM Adv - Deployment issue, need to check and will update
>> KVM eip - 95.7%
>
> Good so it’s running now.
Yes, after the agent fixes this is back up
>
>> VMware Adv - deployment issue, need to check and update
>
Re: BVT report 9/23
Posted by Sebastien Goasguen <ru...@gmail.com>.
> On Sep 24, 2015, at 9:08 PM, Raja Pullela <ra...@citrix.com> wrote:
>
> BVT report 09/23
>
> simulator basic - 30% , earlier runs had 100% pass rate, failures need to be analyzed
What’s earlier ? yesterday, last week, six months ago ?
> Simulator adv - 50%, earlier runs had 100% pass rate, failures need to be analyzed
Same here, what’s earlier ?
> XS basic - 95.3%
> XS Adv - 93.6%
> XS eip - 86.7%
Is this good enough from your view for release, or do you want 100% ?
> KVM basic - 89.4%
> KVM Adv - Deployment issue, need to check and will update
> KVM eip - 95.7%
Good so it’s running now.
> VMware Adv - deployment issue, need to check and update
>
>