You are viewing a plain text version of this content. The canonical link for it is here.
Posted to release-discuss@apache.org by "Hyrum K. Wright" <hy...@mail.utexas.edu> on 2010/07/22 20:37:49 UTC
Release process tooling
I've been thinking a bit lately about the way we can provide better
tools for projects to release. The overarching goal is to provide
tools which reduce the barriers for projects to create high-quality
verified and signed releases. This is a brain dump of my thoughts,
with which I'd like to stimulate some discussion.
There are a lot of bikeshedable components, but generally I'm thinking
about the following workflow:
* RM creates artifact(s)
* RM signs the artifact(s)
* RM registers the artifact(s) using a script on {people,dist,?}.apache.org
* PMC members go to a webapp to download the artifact(s)
* offline, the PMC members verify and sign the artifact(s)
* PMC members then upload signatures through the webapp
* webapp verifies:
a) the signature is valid
b) the signer is authorized to sign the artifact(s) (i.e., is a
member of the PMC)
* RM retrieves the signatures via script or webapp
* RM can then run script to promote the artifact(s), with signatures
and hashes, to the distribution area
Some of this reflects my own bias as the Subversion RM for the past
couple of years, but a lot of it is the result of various
conversations with Paul, Henri and other folks. Aside from the
questions of storage and other minutiae, am I missing anything?
-Hyrum
Re: Release process tooling
Posted by Antonio Petrelli <an...@gmail.com>.
2010/7/23 Brian Fox <br...@infinity.nu>:
> With the exception of the rest of the PMC signing the artifacts,
> everything else you described is currently done automatically by using
> Nexus for Maven artifacts. We could in theory extend it to be able to
> handle the staging of distros as well via a plugin.
I've done something that can be useful in sandbox of Velocity only
using existing plugins, at least in the upload phase:
http://svn.eu.apache.org/repos/asf/velocity/sandbox/maven-reorg/engine/trunk/velocity-engine-assembly/pom.xml
The important parts are:
* creating checksums in the install phase, so they will be deployed in
the local repository;
* skip the Maven deploy (probably optional)
* copying installed assemblies from the local repository into a subdir
of "target"
* upload the content of that subdir to the configured position.
I tried with a snapshot and it seems to work (when activating the
"apache-release" profile).
Antonio
Re: Release process tooling
Posted by Brian Fox <br...@infinity.nu>.
With the exception of the rest of the PMC signing the artifacts,
everything else you described is currently done automatically by using
Nexus for Maven artifacts. We could in theory extend it to be able to
handle the staging of distros as well via a plugin.
On Thu, Jul 22, 2010 at 2:37 PM, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
> I've been thinking a bit lately about the way we can provide better
> tools for projects to release. The overarching goal is to provide
> tools which reduce the barriers for projects to create high-quality
> verified and signed releases. This is a brain dump of my thoughts,
> with which I'd like to stimulate some discussion.
>
> There are a lot of bikeshedable components, but generally I'm thinking
> about the following workflow:
> * RM creates artifact(s)
> * RM signs the artifact(s)
> * RM registers the artifact(s) using a script on {people,dist,?}.apache.org
> * PMC members go to a webapp to download the artifact(s)
> * offline, the PMC members verify and sign the artifact(s)
> * PMC members then upload signatures through the webapp
> * webapp verifies:
> a) the signature is valid
> b) the signer is authorized to sign the artifact(s) (i.e., is a
> member of the PMC)
> * RM retrieves the signatures via script or webapp
> * RM can then run script to promote the artifact(s), with signatures
> and hashes, to the distribution area
>
> Some of this reflects my own bias as the Subversion RM for the past
> couple of years, but a lot of it is the result of various
> conversations with Paul, Henri and other folks. Aside from the
> questions of storage and other minutiae, am I missing anything?
>
> -Hyrum
>
Re: Release process tooling
Posted by Noah Slater <ns...@apache.org>.
For CouchDB:
On 22 Jul 2010, at 19:37, Hyrum K. Wright wrote:
> * RM creates artifact(s)
> * RM signs the artifact(s)
Running `make distsign` does this for me.
> * RM registers the artifact(s) using a script on {people,dist,?}.apache.org
I use `scp` for this, once I have tested the artefact locally.
> * PMC members go to a webapp to download the artifact(s)
> * offline, the PMC members verify and sign the artifact(s)
> * PMC members then upload signatures through the webapp
> * webapp verifies:
> a) the signature is valid
> b) the signer is authorized to sign the artifact(s) (i.e., is a
> member of the PMC)
> * RM retrieves the signatures via script or webapp
I am the only one signing the artefact at the moment.
> * RM can then run script to promote the artifact(s), with signatures
> and hashes, to the distribution area
I have `rsync` in a script called `release.sh` on p.a.o that does this.
Re: Release process tooling
Posted by Paul Querna <pq...@apache.org>.
On Thu, Jul 22, 2010 at 11:37 AM, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
> I've been thinking a bit lately about the way we can provide better
> tools for projects to release. The overarching goal is to provide
> tools which reduce the barriers for projects to create high-quality
> verified and signed releases. This is a brain dump of my thoughts,
> with which I'd like to stimulate some discussion.
>
> There are a lot of bikeshedable components, but generally I'm thinking
> about the following workflow:
> * RM creates artifact(s)
> * RM signs the artifact(s)
for httpd:
<https://svn.apache.org/repos/asf/httpd/site/trunk/tools/release.sh>
> * RM registers the artifact(s) using a script on {people,dist,?}.apache.org
> * PMC members go to a webapp to download the artifact(s)
> * offline, the PMC members verify and sign the artifact(s)
> * PMC members then upload signatures through the webapp
> * webapp verifies:
> a) the signature is valid
> b) the signer is authorized to sign the artifact(s) (i.e., is a
> member of the PMC)
> * RM retrieves the signatures via script or webapp
> * RM can then run script to promote the artifact(s), with signatures
> and hashes, to the distribution area
>
> Some of this reflects my own bias as the Subversion RM for the past
> couple of years, but a lot of it is the result of various
> conversations with Paul, Henri and other folks. Aside from the
> questions of storage and other minutiae, am I missing anything?
I think most of it makes sense. I wonder though if some of this is
best served by a command line tool making http calls -- just so you
can do things like pipe signatures to a local gpg instance for
validation in a more semi-automated way, might be an easier interface
for some tasks than a webapp. Either way, it still relies upon
getting a webapp going, we can make other interfaces for a better user
experience later.
Re: Release process tooling
Posted by Antonio Petrelli <an...@gmail.com>.
2010/7/22 Hyrum K. Wright <hy...@mail.utexas.edu>:
> I've been thinking a bit lately about the way we can provide better
> tools for projects to release. The overarching goal is to provide
> tools which reduce the barriers for projects to create high-quality
> verified and signed releases.
I agree. As a RM of Tiles I noticed that all the process you mentioned
is perfect when talking about releasing Maven artifacts.
But about distributions, a lot of manual work still needs to be done.
And non Maven users (like in native languages projects, like
Subversion) it's a worse situation.
Antonio
Re: Release process tooling
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
On Mon, Jul 26, 2010 at 11:03 AM, Paul Querna <pq...@apache.org> wrote:
> On Thu, Jul 22, 2010 at 11:37 AM, Hyrum K. Wright
> <hy...@mail.utexas.edu> wrote:
>> I've been thinking a bit lately about the way we can provide better
>> tools for projects to release. The overarching goal is to provide
>> tools which reduce the barriers for projects to create high-quality
>> verified and signed releases. This is a brain dump of my thoughts,
>> with which I'd like to stimulate some discussion.
>>
>> There are a lot of bikeshedable components, but generally I'm thinking
>> about the following workflow:
>> * RM creates artifact(s)
>> * RM signs the artifact(s)
>
> for httpd:
> <https://svn.apache.org/repos/asf/httpd/site/trunk/tools/release.sh>
Sure, and Subversion has its own version of a tool to create release
artifacts. I think this bit is largely project-dependent.
>> * RM registers the artifact(s) using a script on {people,dist,?}.apache.org
>> * PMC members go to a webapp to download the artifact(s)
>> * offline, the PMC members verify and sign the artifact(s)
>> * PMC members then upload signatures through the webapp
>> * webapp verifies:
>> a) the signature is valid
>> b) the signer is authorized to sign the artifact(s) (i.e., is a
>> member of the PMC)
>> * RM retrieves the signatures via script or webapp
>> * RM can then run script to promote the artifact(s), with signatures
>> and hashes, to the distribution area
>>
>> Some of this reflects my own bias as the Subversion RM for the past
>> couple of years, but a lot of it is the result of various
>> conversations with Paul, Henri and other folks. Aside from the
>> questions of storage and other minutiae, am I missing anything?
>
> I think most of it makes sense. I wonder though if some of this is
> best served by a command line tool making http calls -- just so you
> can do things like pipe signatures to a local gpg instance for
> validation in a more semi-automated way, might be an easier interface
> for some tasks than a webapp. Either way, it still relies upon
> getting a webapp going, we can make other interfaces for a better user
> experience later.
Either a command-line tool or a webapp. They can both be front-ends
to the same database/script/files/whatever that is tracking the state
of the release artifacts. Maybe that's the thing that needs to be
designed first (or maybe it's as simple as a set of flat files in a
svn repo).
-Hyrum