lessons from releasing 10.2

[Rick Hillegas <Ri...@Sun.COM>]

I have documented some of the lessons which I learned from being release 
manager for 10.2: http://wiki.apache.org/db-derby/DerbyTenTwoPostMortem

I'm hoping that these notes can help us make the release process 
smoother the next time around.

Regards,
-Rick

Re: lessons from releasing 10.2

[Rick Hillegas <Ri...@Sun.COM>]

Hi Andrew,

Thanks for responding to my post-mortem. Some comments follow inline...

Andrew McIntyre wrote:
> On 2/22/07, Rick Hillegas <Ri...@sun.com> wrote:
>> I have documented some of the lessons which I learned from being release
>> manager for 10.2: http://wiki.apache.org/db-derby/DerbyTenTwoPostMortem
>>
>> I'm hoping that these notes can help us make the release process
>> smoother the next time around.
>
> Thanks for the detailed writeup, Rick. I'll try to respond to as much
> of the wiki page as possible. I think a lot of the improvements can be
> tackled by the next release manager as the process of the next release
> is happening.
>
> -----
>
> Big Picture:
>
> 1. There are 27 steps in our release process. We should automate and
> eliminate many of these steps.
>
> 2. The checklist should be reformatted to help people execute the
> loops and conditionals.
>
> 3. Some steps are out of sequence. In particular, some steps cause you
> to start a subtask which should have been initiated a month
> previously.
>
> Some of the steps can be automated or condensed. However, there are
> many steps whose hands-on nature cannot be avoided. Many of these
> involve interaction with the community at large, such as calling for a
> vote for the release on the dev list, and simply can't be automated.
>
> Improving the checklist itself will bring some obvious benefits, and I
> agree that breaking the list into subsections and including a timeline
> would be useful.
>
> -----
>
> Eclipse UI plugin
>
> Particularly: The release targets should be able to build the Eclipse
> UI plugin without outside assistance.
>
> The problem here is that the UI plugin relies on several Eclipse
> classes. As I see it, there are two options:
>
> * stub out the classes / methods needed and include a little 'ui
> plugin spec jar' to compile the UI plugin against.
>
> * uncouple the UI plugin releases from the rest of the Derby release
> process. This may be the more attractive option, especially if there
> are any issues legally with creating the spec jar. This option also
> has the benefit of relieving the manager of a Derby release from
> worrying about the plugin which they may not be able to or have
> interest in creating.
I like the idea that this is not the release manager's responsibility. 
Is this what you had in mind:

1) The release manager declares victory on the release and puts the 
other distributions on the download page.

2) Sometime later, someone else generates the UI plugin. Who is this 
person? Where do they publish the UI plugin. Does someone sign the plugin?
>
> -----
>
> Release ID Generation
>
> I think that the release ID should remain as it is now, in
> release.properties, as a lot of other processes rely on the release ID
> residing in this one location. In response to the two points in this
> section:
>
> 1. The canon-based tests should be converted to JUnit so that the
> release ID can be asserted without changing canons.
>
> The metadata tests are the only tests affected by having the release
> ID appear in their canons. While some of these tests have been
> converted, it looks like the primary test, metadata.java, has not. I'm
> not entirely sure where this stands, but I believe there is more to be
> done here.
>
> 2. The release ID should be a parameter to the master build target.
>
> In a sense, this is what we have today. The parameters for the release
> ID of the build are recorded in release.properties and used throughout
> the rest of the build. If you have specific improvements to the
> current situation, please suggest them.
I was imagining something like this:

ant -DreleaseID=10.3.1.2 buildTheWholeReleaseDocsAndAll

Behind the scenes, this could decompose the release ID and poke it into 
release.properties.
>
> ------
>
> Release Notes
>
> Generating readable and clear release notes and maintaining JIRA
> descriptions is one of those manual steps of wordsmithing and
> cat-herding that, unfortunately, are going to be unavoidable. While
> tools can help to generate the release notes, making them clear is
> necessarily a manual review process.
No doubt about it, there's a lot of manual work here. What I'd like to 
keep to a minimum is the re-generating of release candidates as people 
read the release notes and suggest better wording. In previous releases, 
the release notes were generated at the tail-end of the release process 
when the download page was constructed. Now, however, we bundle the 
release notes with the distributions. This means that the release notes 
have to be generated at the beginning of the release process and the 
community has to provide timely feedback. Here are two scenarios:

1) We always budget for a throwaway first release candidate. This 
candidate contains a draft version of the release notes. Once we collect 
all of the feedback on this draft, we generate a second, "real" release 
candidate.

2) Or...Before generating the first release candidate, we circulate a 
draft of the release notes. We don't generate the first release 
candidate until we finish collecting feedback on this draft.

>
> ------
>
> Master Build Target
>
> 1. There should be a single master build target which drives all of
> the mechanical pieces of building a release distribution.
>
> This is a great idea, and one that I think is doable. I propose that
> we have a new Ant build file, that lives in the top of the derby tree,
> at the same level as the code / docs / site / jars directories, and
> performs the sequence of steps that you list. I'm not sure all of the
> steps you list are fully automatable by Ant, note for example the
> difficulties observed on different platforms in signing the final
> release distributions. Also, I believe choosing when to change release
> IDs is still a step that requires human attention.
>
> ------
>
> Balloon Payments
>
> 1. ... changes accumulated which affected the directory structure and
> completeness of the release distribution. It would have been helpful
> if the release machinery could have been run and verified as [ the
> changes in the trunk ] accumulated.
>
> A nightly release build would be possible with the Ant build file
> described above, which would allow for early detection of these sorts
> of changes.
>
> ------
>
> Brittle targets
>
> 1. There is a lot of cut-and-paste in the packaging targets. ... We
> should eliminate this cut-and-paste so that we can have greater
> confidence in our distributions.
>
> Entirely doable, even easy, in the near term. It's simply a matter of
> defining the contents of the distributions as a set of properties to
> include / exclude and using these in the tarfilesets / zipfilesets.
>
> ------
>
> Maven
>
> I agree that using Maven to publish a release to the Maven repository
> is not a painless affair for non-Maven users. However, I see no reason
> for the Derby community not to control the publishing of its own
> release artifacts to the Apache Maven repository, as a manner of
> community oversight. While not always an obvious venture for the
> release manager, the benefits to the larger Derby community should be
> obvious.
I understand the value in archiving releases. But do we really need 
Maven for this? Maven seemed to get in the way and I had to hack around 
its failures with non-Maven tools--why not use those stable, non-Maven 
tools to begin with?
>
> ------
>
> Did I miss anything? I think the clearest areas for improvement are to
> restructure, improve, and simplify the release checklist, and to have
> a master build file that lives above the code, docs, site, and jars
> trees and which interacts with the build processes in each of those
> subtrees. I think the latter will go a long way to easing the
> technical process of creating a release and with that burden relieved,
> the former will go a long way towards making the process of getting
> from "I'll be the release manager" to actually producing a Derby
> release much easier.
Yes, thank you, that's a great summary.

1) Simplify the checklist.

2) Write a master build target which takes releaseID as a parameter.

3) Cajole the community into providing timely feedback on the release notes.

Thanks!
-Rick
>
> Comments, opinions?
>
> andrew

Re: lessons from releasing 10.2

[Andrew McIntyre <mc...@gmail.com>]

On 2/22/07, Rick Hillegas <Ri...@sun.com> wrote:
> I have documented some of the lessons which I learned from being release
> manager for 10.2: http://wiki.apache.org/db-derby/DerbyTenTwoPostMortem
>
> I'm hoping that these notes can help us make the release process
> smoother the next time around.

Thanks for the detailed writeup, Rick. I'll try to respond to as much
of the wiki page as possible. I think a lot of the improvements can be
tackled by the next release manager as the process of the next release
is happening.

-----

Big Picture:

1. There are 27 steps in our release process. We should automate and
eliminate many of these steps.

2. The checklist should be reformatted to help people execute the
loops and conditionals.

3. Some steps are out of sequence. In particular, some steps cause you
to start a subtask which should have been initiated a month
previously.

Some of the steps can be automated or condensed. However, there are
many steps whose hands-on nature cannot be avoided. Many of these
involve interaction with the community at large, such as calling for a
vote for the release on the dev list, and simply can't be automated.

Improving the checklist itself will bring some obvious benefits, and I
agree that breaking the list into subsections and including a timeline
would be useful.

-----

Eclipse UI plugin

Particularly: The release targets should be able to build the Eclipse
UI plugin without outside assistance.

The problem here is that the UI plugin relies on several Eclipse
classes. As I see it, there are two options:

* stub out the classes / methods needed and include a little 'ui
plugin spec jar' to compile the UI plugin against.

* uncouple the UI plugin releases from the rest of the Derby release
process. This may be the more attractive option, especially if there
are any issues legally with creating the spec jar. This option also
has the benefit of relieving the manager of a Derby release from
worrying about the plugin which they may not be able to or have
interest in creating.

-----

Release ID Generation

I think that the release ID should remain as it is now, in
release.properties, as a lot of other processes rely on the release ID
residing in this one location. In response to the two points in this
section:

1. The canon-based tests should be converted to JUnit so that the
release ID can be asserted without changing canons.

The metadata tests are the only tests affected by having the release
ID appear in their canons. While some of these tests have been
converted, it looks like the primary test, metadata.java, has not. I'm
not entirely sure where this stands, but I believe there is more to be
done here.

2. The release ID should be a parameter to the master build target.

In a sense, this is what we have today. The parameters for the release
ID of the build are recorded in release.properties and used throughout
the rest of the build. If you have specific improvements to the
current situation, please suggest them.

------

Release Notes

Generating readable and clear release notes and maintaining JIRA
descriptions is one of those manual steps of wordsmithing and
cat-herding that, unfortunately, are going to be unavoidable. While
tools can help to generate the release notes, making them clear is
necessarily a manual review process.

------

Master Build Target

1. There should be a single master build target which drives all of
the mechanical pieces of building a release distribution.

This is a great idea, and one that I think is doable. I propose that
we have a new Ant build file, that lives in the top of the derby tree,
at the same level as the code / docs / site / jars directories, and
performs the sequence of steps that you list. I'm not sure all of the
steps you list are fully automatable by Ant, note for example the
difficulties observed on different platforms in signing the final
release distributions. Also, I believe choosing when to change release
IDs is still a step that requires human attention.

------

Balloon Payments

1. ... changes accumulated which affected the directory structure and
completeness of the release distribution. It would have been helpful
if the release machinery could have been run and verified as [ the
changes in the trunk ] accumulated.

A nightly release build would be possible with the Ant build file
described above, which would allow for early detection of these sorts
of changes.

------

Brittle targets

1. There is a lot of cut-and-paste in the packaging targets. ... We
should eliminate this cut-and-paste so that we can have greater
confidence in our distributions.

Entirely doable, even easy, in the near term. It's simply a matter of
defining the contents of the distributions as a set of properties to
include / exclude and using these in the tarfilesets / zipfilesets.

------

Maven

I agree that using Maven to publish a release to the Maven repository
is not a painless affair for non-Maven users. However, I see no reason
for the Derby community not to control the publishing of its own
release artifacts to the Apache Maven repository, as a manner of
community oversight. While not always an obvious venture for the
release manager, the benefits to the larger Derby community should be
obvious.

------

Did I miss anything? I think the clearest areas for improvement are to
restructure, improve, and simplify the release checklist, and to have
a master build file that lives above the code, docs, site, and jars
trees and which interacts with the build processes in each of those
subtrees. I think the latter will go a long way to easing the
technical process of creating a release and with that burden relieved,
the former will go a long way towards making the process of getting
from "I'll be the release manager" to actually producing a Derby
release much easier.

Comments, opinions?

andrew