[SDO Java] next steps in preparing a release

[kelvin goodson <ke...@thegoodsons.org.uk>]

I had an instant messaging chat with Amita, covering issues of release
management, including

license compliance, cutting tags/branches, sample programs, using the
distribution project,  building and deploying a release candidate, pgp
signing, maven staging repositories

appended is the transcript

Kelvin.

9:11:55 AM: Amita A Vadhavkar:is there something else i can do for the
release - like create branch, tag, distribution etc?
9:12:17 AM: kelvin_goodson: sure, lets spend some time thinking it through,
then I'll post a summary to the list
9:13:31 AM: kelvin_goodson: so as soon as you take a branch then theres
double fixing to do, so that's best done once the function is in (I spent
lots of yesterday getting my event notification function done and I'm poised
to commit it)
9:13:59 AM: kelvin_goodson: so there are guidelines somewhere we ought to
look through -- (note after chat:
http://incubator.apache.org/guides/releasemanagement.html)
9:14:11 AM: kelvin_goodson: what are the key points ...
9:14:48 AM: kelvin_goodson: Every downloadable artifact should have license
information that gives the downloader a clear picture of the licensed
content of that artifact
9:15:11 AM: kelvin_goodson: this is true for source archive,  binary archive
and maven downloadable artifacts
9:15:19 AM: kelvin_goodson: we should be OK from previous releases for this
9:15:32 AM: kelvin_goodson: in maven artifacts, the license must be in the
manifest of the jar
9:16:05 AM: kelvin_goodson: the samples could perhaps do with some checking
for spelling mistakes in the commentary that is output
9:16:17 AM: kelvin_goodson: have you run the samples?
9:16:21 AM: Amita A Vadhavkar: yes
9:16:31 AM: kelvin_goodson: I spent quite a bit of time on those in the last
release,  but could have spent more
9:16:44 AM: Amita A Vadhavkar: i will check then in detail now
9:16:46 AM: kelvin_goodson: Can you see ways to improve them?
9:16:48 AM: kelvin_goodson: thanks
9:16:52 AM: Amita A Vadhavkar: yes will do
9:17:12 AM: kelvin_goodson: Are you familiar with the "distribution" project
9:17:25 AM: kelvin_goodson: it is used to build the source and binary
artifacts
9:17:43 AM: Amita A Vadhavkar: partially, have done some work in DAS on it-
create assembly pom etc
9:17:47 AM: kelvin_goodson: in the poms for that project it specifies files
that should go through a variable substitution filter
9:17:49 AM: kelvin_goodson: good
9:18:09 AM: kelvin_goodson: so,  there will be variables to ensure they are
the correct value when we cut a branch/tag
9:18:22 AM: kelvin_goodson: the release process should be ...
9:18:50 AM: kelvin_goodson: export source from tag to local build structure
9:19:02 AM: Amita A Vadhavkar: ?
9:19:05 AM: kelvin_goodson: build right through to source / binary artifacts
9:19:15 AM: kelvin_goodson: will come back to your "?"
9:19:48 AM: kelvin_goodson: take the source artifact and act as if you were
someone who had downloaded the source distro and wanted to build the binary
from it
9:20:11 AM: kelvin_goodson: rebuild everything right through to source /
binary from just the source distro
9:20:19 AM: kelvin_goodson: now going back to your "?"
9:20:51 AM: kelvin_goodson: export is an svn command that checks out source
into a plain source hierarchy,  with no .svn folders etc
9:20:56 AM: kelvin_goodson: is that what you were asking?
9:21:21 AM: Amita A Vadhavkar: let me ask one by one
9:21:25 AM: kelvin_goodson: go ahead
9:21:48 AM: Amita A Vadhavkar: 1) how do we create a branch?
9:22:02 AM: kelvin_goodson: svn copy URL1 URL2
9:22:36 AM: kelvin_goodson: You can do this with tortoise repository
explorer
9:22:46 AM: kelvin_goodson: or command line svn
9:22:52 AM: kelvin_goodson: which would be something like
9:23:28 AM: Amita A Vadhavkar: yes saw that
9:23:40 AM: Amita A Vadhavkar: so creating branch means creating tag too?
9:24:02 AM: kelvin_goodson: svn copy
https://svn.apa..../tuscany/java/sdohttps://svn.apa.../tuscany/branches/java/sdo/1.1-incubatingor
whatever the pattern is
9:24:13 AM: kelvin_goodson: a tag is just a copy of the source that is under
the "tags" folder
9:24:30 AM: kelvin_goodson: svn recognises a path with tags in if i
understand correctly
9:25:10 AM: kelvin_goodson: so in the past, because there has been little
activity in the trunk when I have been trying to stabilize the build I have
not created a branch
9:25:35 AM: kelvin_goodson: I have gone straight to a tag at a late stage
and then made minor changes (i.e. pom versions without SNAPSHOT qualifier)
in the tag
9:25:46 AM: kelvin_goodson: svn warns you when you are editing files in a
tag
9:25:51 AM: kelvin_goodson: but doens't stop you (the tag must be considered
immutable once a release has been made,  and must match the source distro)
9:25:54 AM: Amita A Vadhavkar: ok
9:26:09 AM: Amita A Vadhavkar: so when we freeze on JIRAs I can do the same
9:26:40 AM: kelvin_goodson: there may not be a need to "freeze" on JIRAs,
but we can take a JIRA cut and say thats what gets into the release
9:27:24 AM: Amita A Vadhavkar: 2) export source from tag to local build
strcuture
9:27:30 AM: Amita A Vadhavkar: so i create a tag
9:27:35 AM: kelvin_goodson: yes
9:27:39 AM: Amita A Vadhavkar: and then checkout in my local dir
9:27:49 AM: kelvin_goodson: svn export tagurl local-folder
9:27:54 AM: kelvin_goodson: yes
9:27:57 AM: Amita A Vadhavkar: and then check all distro related and other
poms
9:28:11 AM: Amita A Vadhavkar: to make sure the -Pdistribution works correct
9:28:32 AM: kelvin_goodson: so check the poms before the export (there may
be some iteration)
9:28:41 AM: kelvin_goodson: then yes, try -Pdistribution
9:29:04 AM: kelvin_goodson: at the end of that you will have 4 files under
that target folder of the distribution project
9:29:17 AM: kelvin_goodson: source / binary in zip and .tar.gz formats
9:29:27 AM: Amita A Vadhavkar: ok
9:29:56 AM: kelvin_goodson: so then take a source distro and build from it
(preferably in a clean isolated environment)
9:30:20 AM: kelvin_goodson: good opportunity to switch from windows to linux
if you have the env to do it
9:30:30 AM: Amita A Vadhavkar: that was my next qstn
9:30:35 AM: Amita A Vadhavkar: i dont have linux
9:30:40 AM: Amita A Vadhavkar: its a single OS machine
9:30:48 AM: kelvin_goodson: no prob,  just say so on the list
9:31:03 AM: Amita A Vadhavkar: how do u people do it?
9:31:03 AM: kelvin_goodson: so just rebuild from source distro on your
windows m/c
9:31:27 AM: kelvin_goodson: I have a linux m/c,  also you can boot a machine
from a linux live CD if you want
9:31:44 AM: kelvin_goodson: I also have a bootable USB key with a linux
image on it (pendrivelinux)
9:31:54 AM: kelvin_goodson: but that took me quite a while to get to that
point
9:32:18 AM: Amita A Vadhavkar: there was a JIRA about linux specific failure

9:32:31 AM: Amita A Vadhavkar: some timestamp or something similar
9:32:34 AM: kelvin_goodson: yes, i remember
9:32:47 AM: kelvin_goodson: well,  its in the melting pot with all the
others
9:33:08 AM: kelvin_goodson: it would be good to get it done,  but if no-one
screams then we prioritize it accordingly
9:33:34 AM: kelvin_goodson: on another parallel track (happy to come back to
Qs on the other track) .....
9:33:41 AM: kelvin_goodson: do you have a PGP signing key
9:33:42 AM: kelvin_goodson: ?
9:34:07 AM: Amita A Vadhavkar: no
9:34:20 AM: Amita A Vadhavkar: i saw a .bat file in one of ur jiras
9:34:43 AM: kelvin_goodson: ok,  so you need to get gpg (or some other
implementation of pgp)
9:34:49 AM: kelvin_goodson: can't recall the bat file
9:34:55 AM: Amita A Vadhavkar: buildSDORelease.bat
9:35:11 AM: Amita A Vadhavkar: how to get that
9:35:14 AM: kelvin_goodson: ah yes, I think that was for signing -- I'm
hoping we might be able to do better than that
9:35:20 AM: Amita A Vadhavkar: in jira-1171
9:35:29 AM: kelvin_goodson: DAS has fixed the signing issue I think in the
poms
9:35:41 AM: kelvin_goodson: I have a marker on an email somewhere that
points to the solution
9:36:20 AM: kelvin_goodson: my bat file is just to assist with the long
winded manual signing process,  but I think theres a new maven plugin that
helps here
9:36:29 AM: kelvin_goodson: i'll look up the email (note after chat:
https://issues.apache.org/jira/browse/TUSCANY-1489)
9:37:04 AM: kelvin_goodson: so once you have gpg you need to create a pgp
signing key,  using your apache.org email address
9:37:35 AM: kelvin_goodson: then export it in armoured ascii format (see the
gpg help,  something like "--export -sa")
9:37:49 AM: kelvin_goodson: and add the output text to the KEYS file in svn
9:38:16 AM: kelvin_goodson: also you are supposed to register the key with a
signing repository,  I believe I used mit.edu
9:38:28 AM: Amita A Vadhavkar: ok
9:39:33 AM: Amita A Vadhavkar: where is KEYS in svn?
9:39:41 AM: kelvin_goodson: top level tuscany IIRC
9:39:49 AM: kelvin_goodson: near STATUS
9:40:41 AM: kelvin_goodson: so for now I guess going through the process of
building a release candidate,  and examining it  for LICENSE and NOTICE file
compliance would be good
9:40:46 AM: Amita A Vadhavkar: ok kind of got it
9:40:54 AM: kelvin_goodson: making sure those OSOA 3rd party license headers
are right
9:41:13 AM: Amita A Vadhavkar: how to ensure that?
9:41:26 AM: kelvin_goodson: in your previous note to the ML where you asked
about it
9:41:44 AM: kelvin_goodson: I appended a title of an ML thread where the
resolution to the license issue was contained
9:41:58 AM: kelvin_goodson: if you follow the thread you'll find the updated
license text
9:42:27 AM: Amita A Vadhavkar: ok, i found there apache lic text, will check
again
9:42:41 AM: Amita A Vadhavkar: will be in some response i might have skipped
9:42:48 AM: kelvin_goodson: ok
9:43:20 AM: kelvin_goodson: down the line,  when you get to presenting a
release candidate theres a tricky step,  which I think again DAS/SCA have
made some advances on
9:43:31 AM: kelvin_goodson: it would be good to investigate or ask on the ML
9:43:49 AM: Amita A Vadhavkar: ok
9:43:52 AM: kelvin_goodson: the releae candidate must be posted to a staging
repository
9:44:09 AM: kelvin_goodson: I have used my space on people.apache.org in the
past
9:44:28 AM: kelvin_goodson: you mustn't post the candidate maven artifacts
to the proper apache maven repos
9:44:46 AM: Amita A Vadhavkar: ?
9:44:53 AM: kelvin_goodson: yet people must be able to try running maven
against the candidate artifacts
9:46:16 AM: kelvin_goodson:  re your "?" if I recall correctly a maven repo
is just a folder structure with some conventions that can be accessed with
an http URL,  so you can build a structure like that under your public_html
directory in your people.apache.org account
9:46:55 AM: kelvin_goodson: so what you must do when you ask people to look
at a candidate is to simulate the apache maven repo in your local space
9:47:26 AM: kelvin_goodson: people can update their maven settings.xml file
to include your maven repo on the search path, where it will find the
artifacts of the required version
9:48:40 AM: Amita A Vadhavkar: ok
9:49:10 AM: kelvin_goodson: if you were to post the candidate artifacts (mvn
deploy command) to the proper apache repo then those artifacts with possible
flaws, would be mirrored and hard to remove from general usage (mirroring in
apache incubator releases is a new thing since our last release, so it
brings an extra hazard that inadvertant deployment to the apache repo would
be very difficult to recover from)
9:49:44 AM: kelvin_goodson: like I say I think there are better ways of dong
this than I have done in the past
9:50:18 AM: Amita A Vadhavkar: ok
9:50:25 AM: kelvin_goodson: the main constraint is that the final successful
release candidate archives are the "same" as the ones that are posted to the
apache infra
9:50:47 AM: kelvin_goodson: i.e. the md5sum crc digests are identical
9:51:36 AM: kelvin_goodson: so in the past I have done individual mvn deploy
commands on the release candidate archive files, as per the saved scripts
from previous releases
9:51:47 AM: kelvin_goodson: but I think we can be smarter than that
9:51:55 AM: kelvin_goodson: probably worth asking luciano on the ML
9:52:01 AM: Amita A Vadhavkar: yes
9:52:18 AM: kelvin_goodson: I'll post this chat to tuscany-dev ML
9:52:25 AM: kelvin_goodson: i'll fix up the typos
9:52:31 AM: Amita A Vadhavkar: sure
9:52:37 AM: kelvin_goodson: any other questions?
9:52:43 AM: Amita A Vadhavkar: nothing for now
9:52:48 AM: kelvin_goodson: ok,  thanks
9:52:53 AM: Amita A Vadhavkar: thanks