Release auditing guidelines

[Jukka Zitting <ju...@gmail.com>]

Hi,

We've had a somewhat low turnaround of binding votes in recent
releases, and I suspect one of the reasons for that is that there are
no clearly documented release auditing guidelines even though the
release vote is perhaps the most important decision making process of
the PMC. To correct this I decided to put together some practical
steps to follow when auditing a release. I hope I haven't missed
anything important, please point out any errors or omissions.

Good places to check for more general Apache release guidelines and
background are the Apache Releases FAQ at
http://www.apache.org/dev/release.html and the Incubator Release Guide
at http://incubator.apache.org/guides/releasemanagement.html.

This guide takes a more practical approach with the current Apache
Jackrabbit 1.2.1 release candidate as the example. The Unix commands
I've used to illustrate the steps are just examples, you can use
whatever tools you like that have the same effect.

To start the release audit you should check the release notes included
with the release candidate. Check that it contains the appropriate
information and lists the contents of the release. The release must
always contain the released source code, and may also contain
precompiled binaries

The released source code is the most important part of the release. In
the 1.2.1 release candidate all the released source code is placed in
a single package. To start the audit you need to download this
package.

    $ wget http://people.apache.org/~jukka/jackrabbit/1.2.1/jackrabbit-1.2.1-src.jar

The first thing you should check is the digital signature on the
release package. With gpg the verification happens like this:

    $ wget http://people.apache.org/~jukka/jackrabbit/1.2.1/jackrabbit-1.2.1-src.jar.asc
    $ gpg --verify jackrabbit-1.2.1-src.jar.asc

You need to have my public key in your keyring for the verification to
succeed. If you do not yet have the key, you can import it like this:

    $ wget http://people.apache.org/~jukka/jackrabbit/KEYS
    $ gpg --import KEYS

You should also check that the md5 and sha1 checksums of the release
candidate are correct. Just recompute the checksums and compare them
with the ones I published. Below is how to use openssl to compute the
checksums:

    $ openssl sha1 < jackrabbit-1.2.1-src.jar
    $ openssl md5 < jackrabbit-1.2.1-src.jar

Once you've verified the signatures and checksums, you should check
that the release builds and works as it should. The included unit test
suites cover most of the relevant functionality, and you may also want
to try the built artifacts in whatever test applications you normally
use. If you have the JCR TCK available and configured, you probably
want to run the release candidate through it.

    $ jar xf jackrabbit-1.2.1-src.jar
    $ (cd jackrabbit-1.2.1; mvn install)

Note that the actual build command ("mvn install" in this case) should
be documented in the README.txt file.

Finally and most importantly you should check that all the legal
notices and license information is in place. Check the contents of the
NOTICE.txt, LICENSE.txt, and README.txt files for the appropriate
copyright and license notes. You should also check that the source
files included in the release have the correct license headers.
There's a nice tool called RAT (Release Audit Tool) by Robert Burrel
Donkin that automates the license header checks. Check it out at
http://code.google.com/p/arat/. You can download and run it like this
to get a listing of the files without the Apache license header:

    $ wget http://arat.googlecode.com/files/rat-0.4.1.jar
    $ java -jar rat-0.4.1.jar jackrabbit-1.2.1-src | grep -v ASL | less

Once you're done with the source audit, you can also check the
possible pre-compiled binaries. Check the digital signature and
checksums as described above and try the binaries out if you have an
appropriate test environment. Legally the most important thing is to
check that the binaries contain the appropriate NOTICE.txt and
LICENSE.txt files.

All of this does take some time, but it quickly becomes routine after
you've done the audit for a couple of releases. It also helps if
you've followed the release plans and related commit messages as then
you can just check that the release matches the code in version
control and you already have a pretty good picture of the quality and
contents of the release.

If you find these guidelines useful I'll try to tidy them and put them
up on the Apache Jackrabbit web site for permanent record.

BR,

Jukka Zitting

Re: Release auditing guidelines

[Stefan Guggisberg <st...@gmail.com>]

On 1/22/07, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> We've had a somewhat low turnaround of binding votes in recent
> releases, and I suspect one of the reasons for that is that there are
> no clearly documented release auditing guidelines even though the
> release vote is perhaps the most important decision making process of
> the PMC. To correct this I decided to put together some practical
> steps to follow when auditing a release. I hope I haven't missed
> anything important, please point out any errors or omissions.
>
> Good places to check for more general Apache release guidelines and
> background are the Apache Releases FAQ at
> http://www.apache.org/dev/release.html and the Incubator Release Guide
> at http://incubator.apache.org/guides/releasemanagement.html.
>
> This guide takes a more practical approach with the current Apache
> Jackrabbit 1.2.1 release candidate as the example. The Unix commands
> I've used to illustrate the steps are just examples, you can use
> whatever tools you like that have the same effect.
>
> To start the release audit you should check the release notes included
> with the release candidate. Check that it contains the appropriate
> information and lists the contents of the release. The release must
> always contain the released source code, and may also contain
> precompiled binaries
>
> The released source code is the most important part of the release. In
> the 1.2.1 release candidate all the released source code is placed in
> a single package. To start the audit you need to download this
> package.
>
>     $ wget http://people.apache.org/~jukka/jackrabbit/1.2.1/jackrabbit-1.2.1-src.jar
>
> The first thing you should check is the digital signature on the
> release package. With gpg the verification happens like this:
>
>     $ wget http://people.apache.org/~jukka/jackrabbit/1.2.1/jackrabbit-1.2.1-src.jar.asc
>     $ gpg --verify jackrabbit-1.2.1-src.jar.asc
>
> You need to have my public key in your keyring for the verification to
> succeed. If you do not yet have the key, you can import it like this:
>
>     $ wget http://people.apache.org/~jukka/jackrabbit/KEYS
>     $ gpg --import KEYS
>
> You should also check that the md5 and sha1 checksums of the release
> candidate are correct. Just recompute the checksums and compare them
> with the ones I published. Below is how to use openssl to compute the
> checksums:
>
>     $ openssl sha1 < jackrabbit-1.2.1-src.jar
>     $ openssl md5 < jackrabbit-1.2.1-src.jar
>
> Once you've verified the signatures and checksums, you should check
> that the release builds and works as it should. The included unit test
> suites cover most of the relevant functionality, and you may also want
> to try the built artifacts in whatever test applications you normally
> use. If you have the JCR TCK available and configured, you probably
> want to run the release candidate through it.
>
>     $ jar xf jackrabbit-1.2.1-src.jar
>     $ (cd jackrabbit-1.2.1; mvn install)
>
> Note that the actual build command ("mvn install" in this case) should
> be documented in the README.txt file.
>
> Finally and most importantly you should check that all the legal
> notices and license information is in place. Check the contents of the
> NOTICE.txt, LICENSE.txt, and README.txt files for the appropriate
> copyright and license notes. You should also check that the source
> files included in the release have the correct license headers.
> There's a nice tool called RAT (Release Audit Tool) by Robert Burrel
> Donkin that automates the license header checks. Check it out at
> http://code.google.com/p/arat/. You can download and run it like this
> to get a listing of the files without the Apache license header:
>
>     $ wget http://arat.googlecode.com/files/rat-0.4.1.jar
>     $ java -jar rat-0.4.1.jar jackrabbit-1.2.1-src | grep -v ASL | less
>
> Once you're done with the source audit, you can also check the
> possible pre-compiled binaries. Check the digital signature and
> checksums as described above and try the binaries out if you have an
> appropriate test environment. Legally the most important thing is to
> check that the binaries contain the appropriate NOTICE.txt and
> LICENSE.txt files.
>
> All of this does take some time, but it quickly becomes routine after
> you've done the audit for a couple of releases. It also helps if
> you've followed the release plans and related commit messages as then
> you can just check that the release matches the code in version
> control and you already have a pretty good picture of the quality and
> contents of the release.
>
> If you find these guidelines useful I'll try to tidy them and put them
> up on the Apache Jackrabbit web site for permanent record.

thanks jukka, having such guidelines would certainly be useful.

cheers
stefan

>
> BR,
>
> Jukka Zitting
>