Snapshots for testing (was: fix versions, etc.)

[Andrew McIntyre <mc...@gmail.com>]

On 8/15/07, Jean T. Anderson <jt...@bristowhill.com> wrote:
>
> and here's the actual policy from [1]:
>
> "During the process of developing software and preparing a release,
> various packages are made available to the developer community for
> testing purposes. Do not include any links on the project website that
> might encourage non-developers to download and use nightly builds,
> snapshots, release candidates, or any other similar package. The only
> people who are supposed to know about such packages are the people
> following the dev list (or searching its archives) and thus aware of the
> conditions placed on the package."
>
> So it's fine to post the location of a snapshot to the mail list so
> users can test it. It just isn't fine to make it readily accessible from
> a web site because it might (likely) get downloaded for something other
> than test purposes.

But it's only ok to post to the dev list, and not the user list, which
means only *really* motivated users are going to go to the effort to
look for snapshot mail posted to the dev list to go test. This really
limits the utility of snapshots, since everyone on the dev list is
capable of generating the contents of a specific revision themselves
if they want to do testing.

The counterargument posted in one of the release policy threads
mentioned earlier was that if you really want to get the user
community involved in an alpha or beta test, mark the release
artifacts as such, put it to a proper vote, and if the vote passes,
put it on the mirrors and announce it as a beta release and call for
testing. The quality threshold for a beta release is essentially none,
so a vote to have a beta release will almost necessarily pass, unless
there are legal issues that come up. Having the vote in the archives
shows due diligence to the release process, which is important for any
artifacts that are distributed outside of the development community.

andrew

Re: Snapshots for testing (was: fix versions, etc.)

["Jean T. Anderson" <jt...@bristowhill.com>]

Rick Hillegas wrote:
> Andrew McIntyre wrote: 
...
>> The counterargument posted in one of the release policy threads
>> mentioned earlier was that if you really want to get the user
>> community involved in an alpha or beta test, mark the release
>> artifacts as such, put it to a proper vote, and if the vote passes,
>> put it on the mirrors and announce it as a beta release and call for

I've noticed this is what torque does.

Main point is it has gone through the process to make it a formal (beta)
release.

>> testing. The quality threshold for a beta release is essentially none,
>> so a vote to have a beta release will almost necessarily pass, unless
>> there are legal issues that come up. Having the vote in the archives

right, legal has to be covered.

>> shows due diligence to the release process, which is important for any
>> artifacts that are distributed outside of the development community.
>>
>> andrew
>>    
> I think this is a promising idea. It would be great if we could get
> early alpha-feedback on 10.4 long before we plunge into the release
> proper. The process you describe, however, is awkward for a trunk
> snapshot: The snapshooter has to wait a week for a vote to end before
> posting the snapshot. On a volatile codebase like trunk, a snapshot's
> value degrades rapidly in just a week.

maybe look at the torque archives to see how they specifically have
handled it -- or even ask on torque-dev@db.apache.org.

 -jean

Re: Release Spacing, was: Snapshots for testing

[Rick Hillegas <Ri...@Sun.COM>]

Kathey Marsden wrote:
> Rick Hillegas wrote:
>> Bryan Pendleton wrote:
>>
>> ...
>>> Alternatively, we could have smaller, more frequent releases.
>>>
>>> In my dream world, I'd like to see Derby releases occur about twice
>>> as frequently as they currently do.
>>>
>>> I'm sure this would require streamlining the release process, but
>>> perhaps we could put effort into that.
>>>
>>> thanks,
>>>
>>> bryan
>>>
>> According to our download page, it took 14 months to produce 10.2 and 
>> 10 months to produce 10.3. I also would like to see a shorter, more 
>> regular spacing between feature releases. I think that the community 
>> could tolerate 6 month spacings between feature releases. The 
>> following discipline might help:
>>
> I think maintenance releases are also important.  If we can produce a 
> maintenance release at the three month interval, we can hopefully 
> start to get a handle on our growing bug backlog and deliver fixes 
> more quickly to users. I am hoping for example that our next release 
> will be 10.3.2.0.
>
> Kathey
>
>
>
It would be great if someone could volunteer soon to release manage 
10.3.2.0 three months from now. That might help keep the bug-fix release 
from slipping too far out. I'm worried that the community would be 
exhausted by a feature release which followed too closely on the heels 
of the bug-fix release.

Regards,
-Rick

Re: Release Spacing, was: Snapshots for testing

[Kathey Marsden <km...@sbcglobal.net>]

Rick Hillegas wrote:
> Bryan Pendleton wrote:
>
> ...
>> Alternatively, we could have smaller, more frequent releases.
>>
>> In my dream world, I'd like to see Derby releases occur about twice
>> as frequently as they currently do.
>>
>> I'm sure this would require streamlining the release process, but
>> perhaps we could put effort into that.
>>
>> thanks,
>>
>> bryan
>>
> According to our download page, it took 14 months to produce 10.2 and 
> 10 months to produce 10.3. I also would like to see a shorter, more 
> regular spacing between feature releases. I think that the community 
> could tolerate 6 month spacings between feature releases. The 
> following discipline might help:
>
I think maintenance releases are also important.  If we can produce a 
maintenance release at the three month interval, we can hopefully start 
to get a handle on our growing bug backlog and deliver fixes more 
quickly to users. I am hoping for example that our next release will be 
10.3.2.0.

Kathey

Release Spacing, was: Snapshots for testing

[Rick Hillegas <Ri...@Sun.COM>]

Bryan Pendleton wrote:

...
> Alternatively, we could have smaller, more frequent releases.
>
> In my dream world, I'd like to see Derby releases occur about twice
> as frequently as they currently do.
>
> I'm sure this would require streamlining the release process, but
> perhaps we could put effort into that.
>
> thanks,
>
> bryan
>
According to our download page, it took 14 months to produce 10.2 and 10 
months to produce 10.3. I also would like to see a shorter, more regular 
spacing between feature releases. I think that the community could 
tolerate 6 month spacings between feature releases. The following 
discipline might help:

1) As soon as the community vets a feature release, someone should 
volunteer to produce the next feature release in 6 months. That boosts 
our confidence that someone will actually be willing to carry the burden 
when the time comes.

2) Knowing that release date, the community can plan. Contributors can 
make sure that half-finished features are disabled or otherwise not 
exposed when we get into the home-stretch.

And I agree that we should keep investing in improvements to our release 
process. A less onerous process would make it easier to recruit the 
volunteer needed in step (1).

Regards,
-Rick

Re: Snapshots for testing

[Bryan Pendleton <bp...@amberpoint.com>]

> I think this is a promising idea. It would be great if we could get 
> early alpha-feedback on 10.4 long before we plunge into the release 
> proper. 

Alternatively, we could have smaller, more frequent releases.

In my dream world, I'd like to see Derby releases occur about twice
as frequently as they currently do.

I'm sure this would require streamlining the release process, but
perhaps we could put effort into that.

thanks,

bryan

Re: Snapshots for testing (was: fix versions, etc.)

[Rick Hillegas <Ri...@Sun.COM>]

Andrew McIntyre wrote:
> On 8/15/07, Jean T. Anderson <jt...@bristowhill.com> wrote:
>   
>> and here's the actual policy from [1]:
>>
>> "During the process of developing software and preparing a release,
>> various packages are made available to the developer community for
>> testing purposes. Do not include any links on the project website that
>> might encourage non-developers to download and use nightly builds,
>> snapshots, release candidates, or any other similar package. The only
>> people who are supposed to know about such packages are the people
>> following the dev list (or searching its archives) and thus aware of the
>> conditions placed on the package."
>>
>> So it's fine to post the location of a snapshot to the mail list so
>> users can test it. It just isn't fine to make it readily accessible from
>> a web site because it might (likely) get downloaded for something other
>> than test purposes.
>>     
>
> But it's only ok to post to the dev list, and not the user list, which
> means only *really* motivated users are going to go to the effort to
> look for snapshot mail posted to the dev list to go test. This really
> limits the utility of snapshots, since everyone on the dev list is
> capable of generating the contents of a specific revision themselves
> if they want to do testing.
>   
I can still see some value in posting a snapshot for developers--at 
least until we streamline our release processes. Right now, generating a 
snapshot is a bit tricky and I think a lot of developers would be 
relieved if a battle-scarred veteran performed this chore.
> The counterargument posted in one of the release policy threads
> mentioned earlier was that if you really want to get the user
> community involved in an alpha or beta test, mark the release
> artifacts as such, put it to a proper vote, and if the vote passes,
> put it on the mirrors and announce it as a beta release and call for
> testing. The quality threshold for a beta release is essentially none,
> so a vote to have a beta release will almost necessarily pass, unless
> there are legal issues that come up. Having the vote in the archives
> shows due diligence to the release process, which is important for any
> artifacts that are distributed outside of the development community.
>
> andrew
>   
I think this is a promising idea. It would be great if we could get 
early alpha-feedback on 10.4 long before we plunge into the release 
proper. The process you describe, however, is awkward for a trunk 
snapshot: The snapshooter has to wait a week for a vote to end before 
posting the snapshot. On a volatile codebase like trunk, a snapshot's 
value degrades rapidly in just a week.