You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by drew <dr...@baseanswers.com> on 2012/08/17 17:55:01 UTC
Extend release vote till after testing? (was: Re: [QA Report]AOO
3.4.1 RC2 rev 1372282 Test Report(w/o long run testing))
Hi,
My recollection is that the email for voting on the release said the
vote would run till tomorrow - but it sounds like formal testing is not
finished till Monday.
Should the final vote not wait till after the testing is finished?
Thanks
//drew
On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
> We did RC build test against RC2 rev. 1372282. Here is the report
> 1. BVT passed [1]
> 2. PVT is done [2]
> 3. Automation FVT passed [3]
> 4. Installation test complete [4]
> 5. General testing is done [5]
> 6. Native build testing [6]
>
> We are doing long run testing this weekend, and I'll bring the report next
> Monday.
>
> [1]
> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
> [2]
> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
> [3]
> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
> [4] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
> [5] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
> [6]
> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
Re: Extend release vote till after testing? (was: Re: [QA Report]AOO
3.4.1 RC2 rev 1372282 Test Report(w/o long run testing))
Posted by Ji Yan <ya...@gmail.com>.
We did manual and automation test against the RC build. From the test
result, it looks good enough to support vote +1. Long run testing will keep
running over 130 hours, for what I can tell now, the test runs smoothly, we
may need another 60 hours to complete the whole test. But I don't see any
reason to demote this RC.(Anyway RC vote is already closed) If there is any
critical issue found, we will speak out in time.
2012/8/18 Rob Weir <ro...@apache.org>
> On Fri, Aug 17, 2012 at 12:07 PM, drew <dr...@baseanswers.com> wrote:
> > On Fri, 2012-08-17 at 12:02 -0400, Rob Weir wrote:
> >> On Fri, Aug 17, 2012 at 11:55 AM, drew <dr...@baseanswers.com> wrote:
> >> > Hi,
> >> >
> >> > My recollection is that the email for voting on the release said the
> >> > vote would run till tomorrow - but it sounds like formal testing is
> not
> >> > finished till Monday.
> >> >
> >> > Should the final vote not wait till after the testing is finished?
> >> >
> >>
> >> Not necessarily. Just because someone calls their tests "formal" does
> >> not mean we need to wait for them. My tests are just as valid as your
> >> test or anyone else's tests.
> >>
> >> I think we can move ahead, if the PPMC vote passes, and start the IPMC
> >> process. Of course, if at any time someone finds a showstopper
> >> issue, we can cancel the vote, at any stage. Even if a showstopper
> >> issue is found after the IPMC vote and minutes before we announce, we
> >> can still recall the release. There is no train on autopilot here.
> >>
> >
> > Ah ok - and your opinion on the specific of this question about the
> > current release is - Yes, wait for Monday or No, don't wait?
> >
>
> My opinion:
>
> 1) There is no limit to how much testing we can do on a release. Any
> decision to end testing is arbitrary. We could always take more time
> and do more kinds of tests, in more variations.
>
> 2) The proposal of this release candidate and the timing of it should
> not be surprising to any member of the community. It has been the
> primary topic of conversation on this list for nearly a month.
>
> 3) Just as it is the responsibility of every programmer to get their
> code submitted in time for the RC, and for every translator to get
> their translations submitted in time for the RC, it is the
> responsibility of ever tester to get their tests done in time.
>
> 4) If anyone thought that there were critical tests that must be
> performed, but these tests could not be performed before the release
> vote ended, then they should have spoken up quite a long time ago.
>
> 5) We're currently holding back fixes from our users, some quite
> critical, by delaying the release.
>
> We gain absolutely nothing by delaying the vote. If we delay and
> nothing is found then we've wasted time. If we don't delay and we
> find a bug then we go back and cut a new Release Candidate. But we
> should we assume that further testing is going to find a show stopping
> bug? That doesn't make sense to me.
>
> > I would wait.
> >
>
> And what if I said I had even more tests that I wanted to run, but
> they would take two weeks more?
>
> -Rob
>
> > Thanks
> >
> > ps - just forget I used the word formal in the question, if that helps.
> >
> >> -Rob
> >>
> >> > Thanks
> >> >
> >> > //drew
> >> >
> >> > On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
> >> >> We did RC build test against RC2 rev. 1372282. Here is the report
> >> >> 1. BVT passed [1]
> >> >> 2. PVT is done [2]
> >> >> 3. Automation FVT passed [3]
> >> >> 4. Installation test complete [4]
> >> >> 5. General testing is done [5]
> >> >> 6. Native build testing [6]
> >> >>
> >> >> We are doing long run testing this weekend, and I'll bring the
> report next
> >> >> Monday.
> >> >>
> >> >> [1]
> >> >>
> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
> >> >> [2]
> >> >>
> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
> >> >> [3]
> >> >>
> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
> >> >> [4]
> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
> >> >> [5]
> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
> >> >> [6]
> >> >>
> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
> >> >
> >> >
> >>
> >
> >
>
--
Thanks & Best Regards, Yan Ji
Re: Extend release vote till after testing? (was: Re: [QA Report]AOO
3.4.1 RC2 rev 1372282 Test Report(w/o long run testing))
Posted by Rob Weir <ro...@apache.org>.
On Fri, Aug 17, 2012 at 12:07 PM, drew <dr...@baseanswers.com> wrote:
> On Fri, 2012-08-17 at 12:02 -0400, Rob Weir wrote:
>> On Fri, Aug 17, 2012 at 11:55 AM, drew <dr...@baseanswers.com> wrote:
>> > Hi,
>> >
>> > My recollection is that the email for voting on the release said the
>> > vote would run till tomorrow - but it sounds like formal testing is not
>> > finished till Monday.
>> >
>> > Should the final vote not wait till after the testing is finished?
>> >
>>
>> Not necessarily. Just because someone calls their tests "formal" does
>> not mean we need to wait for them. My tests are just as valid as your
>> test or anyone else's tests.
>>
>> I think we can move ahead, if the PPMC vote passes, and start the IPMC
>> process. Of course, if at any time someone finds a showstopper
>> issue, we can cancel the vote, at any stage. Even if a showstopper
>> issue is found after the IPMC vote and minutes before we announce, we
>> can still recall the release. There is no train on autopilot here.
>>
>
> Ah ok - and your opinion on the specific of this question about the
> current release is - Yes, wait for Monday or No, don't wait?
>
My opinion:
1) There is no limit to how much testing we can do on a release. Any
decision to end testing is arbitrary. We could always take more time
and do more kinds of tests, in more variations.
2) The proposal of this release candidate and the timing of it should
not be surprising to any member of the community. It has been the
primary topic of conversation on this list for nearly a month.
3) Just as it is the responsibility of every programmer to get their
code submitted in time for the RC, and for every translator to get
their translations submitted in time for the RC, it is the
responsibility of ever tester to get their tests done in time.
4) If anyone thought that there were critical tests that must be
performed, but these tests could not be performed before the release
vote ended, then they should have spoken up quite a long time ago.
5) We're currently holding back fixes from our users, some quite
critical, by delaying the release.
We gain absolutely nothing by delaying the vote. If we delay and
nothing is found then we've wasted time. If we don't delay and we
find a bug then we go back and cut a new Release Candidate. But we
should we assume that further testing is going to find a show stopping
bug? That doesn't make sense to me.
> I would wait.
>
And what if I said I had even more tests that I wanted to run, but
they would take two weeks more?
-Rob
> Thanks
>
> ps - just forget I used the word formal in the question, if that helps.
>
>> -Rob
>>
>> > Thanks
>> >
>> > //drew
>> >
>> > On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
>> >> We did RC build test against RC2 rev. 1372282. Here is the report
>> >> 1. BVT passed [1]
>> >> 2. PVT is done [2]
>> >> 3. Automation FVT passed [3]
>> >> 4. Installation test complete [4]
>> >> 5. General testing is done [5]
>> >> 6. Native build testing [6]
>> >>
>> >> We are doing long run testing this weekend, and I'll bring the report next
>> >> Monday.
>> >>
>> >> [1]
>> >> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
>> >> [2]
>> >> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
>> >> [3]
>> >> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
>> >> [4] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
>> >> [5] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
>> >> [6]
>> >> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
>> >
>> >
>>
>
>
Re: Extend release vote till after testing?
Posted by drew jensen <dr...@gmail.com>.
On Fri, 2012-08-17 at 18:26 +0200, Jürgen Schmidt wrote:
> On 8/17/12 6:07 PM, drew wrote:
> > On Fri, 2012-08-17 at 12:02 -0400, Rob Weir wrote:
> >> On Fri, Aug 17, 2012 at 11:55 AM, drew <dr...@baseanswers.com> wrote:
> >>> Hi,
> >>>
> >>> My recollection is that the email for voting on the release said the
> >>> vote would run till tomorrow - but it sounds like formal testing is not
> >>> finished till Monday.
> >>>
> >>> Should the final vote not wait till after the testing is finished?
> >>>
> >>
> >> Not necessarily. Just because someone calls their tests "formal" does
> >> not mean we need to wait for them. My tests are just as valid as your
> >> test or anyone else's tests.
> >>
> >> I think we can move ahead, if the PPMC vote passes, and start the IPMC
> >> process. Of course, if at any time someone finds a showstopper
> >> issue, we can cancel the vote, at any stage. Even if a showstopper
> >> issue is found after the IPMC vote and minutes before we announce, we
> >> can still recall the release. There is no train on autopilot here.
> >>
> >
> > Ah ok - and your opinion on the specific of this question about the
> > current release is - Yes, wait for Monday or No, don't wait?
> >
> > I would wait.
>
> no need to wait, if you feel comfortable with your testing/verification
> cast your vote.
>
> Juergen
>
Well, I thought given the head email here and what I was seeing
regarding the RAT scan that, in this specific case, it might make sense
to push out the extra 24 hours - but I don't feel strongly about it.
//drew
>
> >
> > Thanks
> >
> > ps - just forget I used the word formal in the question, if that helps.
> >
> >> -Rob
> >>
> >>> Thanks
> >>>
> >>> //drew
> >>>
> >>> On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
> >>>> We did RC build test against RC2 rev. 1372282. Here is the report
> >>>> 1. BVT passed [1]
> >>>> 2. PVT is done [2]
> >>>> 3. Automation FVT passed [3]
> >>>> 4. Installation test complete [4]
> >>>> 5. General testing is done [5]
> >>>> 6. Native build testing [6]
> >>>>
> >>>> We are doing long run testing this weekend, and I'll bring the report next
> >>>> Monday.
> >>>>
> >>>> [1]
> >>>> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
> >>>> [2]
> >>>> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
> >>>> [3]
> >>>> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
> >>>> [4] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
> >>>> [5] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
> >>>> [6]
> >>>> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
> >>>
> >>>
> >>
> >
> >
>
Re: Extend release vote till after testing?
Posted by Jürgen Schmidt <jo...@gmail.com>.
On 8/17/12 6:07 PM, drew wrote:
> On Fri, 2012-08-17 at 12:02 -0400, Rob Weir wrote:
>> On Fri, Aug 17, 2012 at 11:55 AM, drew <dr...@baseanswers.com> wrote:
>>> Hi,
>>>
>>> My recollection is that the email for voting on the release said the
>>> vote would run till tomorrow - but it sounds like formal testing is not
>>> finished till Monday.
>>>
>>> Should the final vote not wait till after the testing is finished?
>>>
>>
>> Not necessarily. Just because someone calls their tests "formal" does
>> not mean we need to wait for them. My tests are just as valid as your
>> test or anyone else's tests.
>>
>> I think we can move ahead, if the PPMC vote passes, and start the IPMC
>> process. Of course, if at any time someone finds a showstopper
>> issue, we can cancel the vote, at any stage. Even if a showstopper
>> issue is found after the IPMC vote and minutes before we announce, we
>> can still recall the release. There is no train on autopilot here.
>>
>
> Ah ok - and your opinion on the specific of this question about the
> current release is - Yes, wait for Monday or No, don't wait?
>
> I would wait.
no need to wait, if you feel comfortable with your testing/verification
cast your vote.
Juergen
>
> Thanks
>
> ps - just forget I used the word formal in the question, if that helps.
>
>> -Rob
>>
>>> Thanks
>>>
>>> //drew
>>>
>>> On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
>>>> We did RC build test against RC2 rev. 1372282. Here is the report
>>>> 1. BVT passed [1]
>>>> 2. PVT is done [2]
>>>> 3. Automation FVT passed [3]
>>>> 4. Installation test complete [4]
>>>> 5. General testing is done [5]
>>>> 6. Native build testing [6]
>>>>
>>>> We are doing long run testing this weekend, and I'll bring the report next
>>>> Monday.
>>>>
>>>> [1]
>>>> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
>>>> [2]
>>>> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
>>>> [3]
>>>> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
>>>> [4] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
>>>> [5] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
>>>> [6]
>>>> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
>>>
>>>
>>
>
>
Re: Extend release vote till after testing? (was: Re: [QA
Report]AOO 3.4.1 RC2 rev 1372282 Test Report(w/o long run testing))
Posted by drew <dr...@baseanswers.com>.
On Fri, 2012-08-17 at 12:02 -0400, Rob Weir wrote:
> On Fri, Aug 17, 2012 at 11:55 AM, drew <dr...@baseanswers.com> wrote:
> > Hi,
> >
> > My recollection is that the email for voting on the release said the
> > vote would run till tomorrow - but it sounds like formal testing is not
> > finished till Monday.
> >
> > Should the final vote not wait till after the testing is finished?
> >
>
> Not necessarily. Just because someone calls their tests "formal" does
> not mean we need to wait for them. My tests are just as valid as your
> test or anyone else's tests.
>
> I think we can move ahead, if the PPMC vote passes, and start the IPMC
> process. Of course, if at any time someone finds a showstopper
> issue, we can cancel the vote, at any stage. Even if a showstopper
> issue is found after the IPMC vote and minutes before we announce, we
> can still recall the release. There is no train on autopilot here.
>
Ah ok - and your opinion on the specific of this question about the
current release is - Yes, wait for Monday or No, don't wait?
I would wait.
Thanks
ps - just forget I used the word formal in the question, if that helps.
> -Rob
>
> > Thanks
> >
> > //drew
> >
> > On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
> >> We did RC build test against RC2 rev. 1372282. Here is the report
> >> 1. BVT passed [1]
> >> 2. PVT is done [2]
> >> 3. Automation FVT passed [3]
> >> 4. Installation test complete [4]
> >> 5. General testing is done [5]
> >> 6. Native build testing [6]
> >>
> >> We are doing long run testing this weekend, and I'll bring the report next
> >> Monday.
> >>
> >> [1]
> >> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
> >> [2]
> >> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
> >> [3]
> >> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
> >> [4] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
> >> [5] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
> >> [6]
> >> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
> >
> >
>
Re: Extend release vote till after testing? (was: Re: [QA Report]AOO
3.4.1 RC2 rev 1372282 Test Report(w/o long run testing))
Posted by Rob Weir <ro...@apache.org>.
On Fri, Aug 17, 2012 at 11:55 AM, drew <dr...@baseanswers.com> wrote:
> Hi,
>
> My recollection is that the email for voting on the release said the
> vote would run till tomorrow - but it sounds like formal testing is not
> finished till Monday.
>
> Should the final vote not wait till after the testing is finished?
>
Not necessarily. Just because someone calls their tests "formal" does
not mean we need to wait for them. My tests are just as valid as your
test or anyone else's tests.
I think we can move ahead, if the PPMC vote passes, and start the IPMC
process. Of course, if at any time someone finds a showstopper
issue, we can cancel the vote, at any stage. Even if a showstopper
issue is found after the IPMC vote and minutes before we announce, we
can still recall the release. There is no train on autopilot here.
-Rob
> Thanks
>
> //drew
>
> On Fri, 2012-08-17 at 23:33 +0800, Ji Yan wrote:
>> We did RC build test against RC2 rev. 1372282. Here is the report
>> 1. BVT passed [1]
>> 2. PVT is done [2]
>> 3. Automation FVT passed [3]
>> 4. Installation test complete [4]
>> 5. General testing is done [5]
>> 6. Native build testing [6]
>>
>> We are doing long run testing this weekend, and I'll bring the report next
>> Monday.
>>
>> [1]
>> http://wiki.openoffice.org/wiki/QA/Report/BVT#BVT_Report_for_AOO3.4.1_Branch_r1372282
>> [2]
>> http://wiki.openoffice.org/wiki/QA/AOO341_PVTResult#PVT_report_on_3.4.1_RC1_r1372282
>> [3]
>> http://wiki.openoffice.org/wiki/QA/Report/FVT#FVT_Report_for_AOO3.4.1_Branch_r1372282
>> [4] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Installer_2
>> [5] http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Rev._1372282
>> [6]
>> http://wiki.openoffice.org/wiki/QA/AOO341_RC_TestResult#Native_Build_Testing
>
>