You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cordova.apache.org by Jukka Zitting <ju...@gmail.com> on 2012/09/20 17:50:06 UTC
Lower-overhead release process
Hi,
We've gone through a couple of iterations of the release process, but
I still think that there's some room for improvement on better fitting
the Apache requirements with the way the Cordova team manages
releases. Here's one idea that I briefly discussed with Andre and Dave
when they dropped by in Basel today.
A distinctive feature of the Cordova release process is the tagging
phase where people working on different platforms each tag the
respective repositories. Currently these tags are then used by the
release manager to put together the final release candidate. I think
we could simplify this two-step process by asking everyone who tags a
repository to also produce a release archive for that tag. With a
cordova-release.sh script like the one I've included at the end of
this message, this would be as simple as:
$ cd incubator-cordova-android
$ cordova-release.sh 2.2.0
As a result you'll get a signed tag and a related signed and
checksummed release tarball:
$ git tag -v 2.2.0
object 11dbf05c8452d045a25a59071201b963c52c8e83
type commit
tag 2.2.0
tagger Jukka Zitting <ju...@apache.org> 1348155222 +0200
incubator-cordova-android version 2.2.0
gpg: Signature made Thu 20 Sep 2012 17:33:42 CEST using DSA key ID A355A63E
gpg: Good signature from "Jukka Zitting <ju...@apache.org>"
$ ls *.tar.gz*
incubator-cordova-android-2.2.0.tar.gz
incubator-cordova-android-2.2.0.tar.gz.asc
incubator-cordova-android-2.2.0.tar.gz.md5
incubator-cordova-android-2.2.0.tar.gz.sha
You can then push the tag and upload the tarball to people.apache.org
or wherever is most convenient. Also these steps could be automated.
The responsibilities of the release manager are then to coordinate
this effort, to collect all produced tarballs, to conduct the release
vote once all the components are tagged, and finally to make the
combined release available to the world.
WDYT?
BR,
Jukka Zitting
----
$ cat cordova-release.sh
#!/bin/sh
TAG="$1"
if [ -z "$TAG" ]; then
echo "usage: cordova-release.sh x.y.z" 1>&2
exit 1
fi
DIR=`git config remote.origin.url | sed -e 's/.*\/\(.*\)\.git/\1/'`
if [ -z "$DIR" ]; then
echo "error: Unable to determine the repository name." 1>&2
exit 2
fi
TGZ="$DIR-$TAG.tar.gz"
if [ -e "$TGZ" ]; then
echo "error: $TGZ already exists." 1>&2
exit 3
fi
set -e
git tag -s -m "$DIR version $TAG" "$TAG"
git archive --output="$TGZ" --prefix="$DIR-$TAG/" "$TAG"
gpg --armor --output "$TGZ.asc" --detach-sig "$TGZ"
gpg --print-md MD5 "$TGZ" > "$TGZ.md5"
gpg --print-md SHA512 "$TGZ" > "$TGZ.sha"
exit 0
Re: Lower-overhead release process
Posted by Dave Johnson <da...@gmail.com>.
Oh and Jukka also mentioned about what the output of the release looks like
wrt how developers can use our code in their own software (thinking
projects like PhoneGap, worklight, Salesforce hybrid SDK, facebook - jokes!
etc). For example, there's currently no clear way for developers to take
our code and use the "CordovaView" to embed content in their otherwise
native apps. This is probably a different thread / issue for another day
but figured I'd raise it anyhow.
- d
On Friday, September 21, 2012, Dave Johnson wrote:
> +1 to CI
>
> On Friday, September 21, 2012, Jukka Zitting wrote:
>
>> Hi,
>>
>> On Thu, Sep 20, 2012 at 8:57 PM, Filip Maj <fi...@adobe.com> wrote:
>> > Talking with Anis, CI would solve this problem..
>>
>> My favorite topic! There are quite a few CI resources available at
>> Apache (see http://ci.apache.org/) and elsewhere (like Travis,
>> http://travis-ci.org/), though I'm not sure how well suited they are
>> to testing mobile code.
>>
>> Alternatively, are there already some CI builds for PhoneGap that we
>> could leverage also for Cordova? The PhoneGap Build infrastructure
>> sounds like it could be adapted also for this purpose.
>>
>> BR,
>>
>> Jukka Zitting
>>
>
Re: Lower-overhead release process
Posted by Dave Johnson <da...@gmail.com>.
+1 to CI
On Friday, September 21, 2012, Jukka Zitting wrote:
> Hi,
>
> On Thu, Sep 20, 2012 at 8:57 PM, Filip Maj <fil@adobe.com <javascript:;>>
> wrote:
> > Talking with Anis, CI would solve this problem..
>
> My favorite topic! There are quite a few CI resources available at
> Apache (see http://ci.apache.org/) and elsewhere (like Travis,
> http://travis-ci.org/), though I'm not sure how well suited they are
> to testing mobile code.
>
> Alternatively, are there already some CI builds for PhoneGap that we
> could leverage also for Cordova? The PhoneGap Build infrastructure
> sounds like it could be adapted also for this purpose.
>
> BR,
>
> Jukka Zitting
>
Re: Lower-overhead release process
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Sep 20, 2012 at 8:57 PM, Filip Maj <fi...@adobe.com> wrote:
> Talking with Anis, CI would solve this problem..
My favorite topic! There are quite a few CI resources available at
Apache (see http://ci.apache.org/) and elsewhere (like Travis,
http://travis-ci.org/), though I'm not sure how well suited they are
to testing mobile code.
Alternatively, are there already some CI builds for PhoneGap that we
could leverage also for Cordova? The PhoneGap Build infrastructure
sounds like it could be adapted also for this purpose.
BR,
Jukka Zitting
Re: Lower-overhead release process
Posted by Michael Brooks <mi...@michaelbrooks.ca>.
On my first read, I thought Jukka was suggesting that each platform should
develop on its own track. For example, Android may be at tag 4.0.3, iOS at
3.1.2, etc. Then at release time, we would cut a "2.2.0" from each of their
latest tags. However, I totally misunderstood.
When looking at the current 2.1.0 release candidate my first
> impression is that there's no build script and the only instructions
> on how to build this thing is to "change directories to the platform
> that you wish to build for and read the README file." And to get to
> those platform sources I first need to unpack them to a separate
> directory.
I agree that the release packaging and building can be cleaned up. Like
many components of this project, it will continue to improve on each
iteration. The notion of cutting a source release is new to most of the
Apache Cordova team and hopefully we'll streamline it a little more on each
release.
The underlying issue is that this project has a lot of shared resources.
Fil has nailed the main issue. Each platform release is dependent on a few
shared projects. If any of the platforms encounter a bug that must be fixed
in the shared resource, then the platform release must be restarted. CI is
a big factor is helping to reduce the chance of a release-restart.
$ cd incubator-cordova-android
> $ cordova-release.sh 2.2.0
I do like this suggestion, although the signed git tag may be too big of a
leap to start off with. I'm not suggesting that we adopt the script, but
it's worth thinking about in the context of our coho release tool. The idea
is that a source release is blind to the platform and simply cuts the
source release from a given tag (perhaps running an audit for common
oversights).
Michael
On Thu, Sep 20, 2012 at 11:57 AM, Filip Maj <fi...@adobe.com> wrote:
> Talking with Anis, CI would solve this problem..
>
> On 9/20/12 11:53 AM, "Filip Maj" <fi...@adobe.com> wrote:
>
> >I agree with Joe in that the specific, automated steps to create a
> >sub-project's release won't help in streamlining the release process for
> >this project. Appreciate your efforts to help in this regard though,
> >Jukka.
> >
> >The underlying issue is that this project has a lot of shared resources.
> >The javascript, example "hello world" app and the test suite are
> >standalone projects but are used by all platform implementations. The crux
> >of the slow release process is this: the shared projects need to be tagged
> >before the platform implementations can be. If we find a problem in the
> >shared projects, then all platforms need a retagging.
> >
> >If the above problem can be solved, then the release flow will improve.
> >
> >Perhaps this is more a matter of diligence and testing than a problem with
> >the release steps themselves. If we are better at catching issues with
> >shared sub-projects on all platforms, then the retag cycle (ideally) would
> >be eliminated.
> >
> >Either way, there is ceremony and process involved. A lot of testing. Many
> >devices. Lots of platforms. Communication and a bit of planning can help
> >here. A week before a potential tag we could create an issue to track
> >progress of testing JS+test suite+hello world sub-projects across all
> >platforms. Only once all of those tasks are done can we with confidence
> >say that the shared projects are stable and we can copy them into the
> >platform implementations that we ship and tag those.
> >
> >Thoughts?
> >
> >On 9/20/12 11:32 AM, "Jukka Zitting" <ju...@gmail.com> wrote:
> >
> >>Hi,
> >>
> >>On Thu, Sep 20, 2012 at 6:45 PM, Joe Bowser <bo...@gmail.com> wrote:
> >>> I know, I'm just saying that this has the same effect as releasing the
> >>> components independently. I'm not convinced that this would reduce
> >>> overhead when it comes to releases.
> >>
> >>OK, fair enough.
> >>
> >>> Also, how would this make us conform to the Apache requirements
> >>> better?
> >>
> >>Not strict requirements as such (the current release package is IMHO
> >>fine from a policy perspective), but rather a more pragmatic point of
> >>view that underlies the Apache policies.
> >>
> >>The release guide [1] speaks of releases being "in the form of the
> >>source materials needed to make changes to the software", which in
> >>practice means that it's a good idea for the structure of the release
> >>to be as close as possible to the structure of the source in the
> >>version control system. The purpose of cutting a source release is not
> >>to satisfy an abstract policy but rather to produce something that
> >>downstream developers can easily use and modify to best match their
> >>needs.
> >>
> >>When looking at the current 2.1.0 release candidate my first
> >>impression is that there's no build script and the only instructions
> >>on how to build this thing is to "change directories to the platform
> >>that you wish to build for and read the README file." And to get to
> >>those platform sources I first need to unpack them to a separate
> >>directory. I might be wrong, but it seems to me that most people would
> >>rather simply start from a tag of a relevant platfrom repository
> >>instead of building the release candidate. If that assumption is
> >>correct, then I think it would be a good idea for the release
> >>structure to better match the expected access patterns.
> >>
> >>The main point I'm trying to address here is the grumbling I see about
> >>the whole release process and its inefficiency. If the outcome of the
> >>release process isn't something that's useful, then we're doing
> >>something wrong and should fix that. If the process itself is too
> >>complicated or otherwise takes too long, we should fix that too. I'd
> >>love to hear also other ideas on how that could be done.
> >>
> >>[1] http://www.apache.org/dev/release.html#what
> >>
> >>BR,
> >>
> >>Jukka Zitting
> >
>
>
Re: Lower-overhead release process
Posted by Filip Maj <fi...@adobe.com>.
Talking with Anis, CI would solve this problem..
On 9/20/12 11:53 AM, "Filip Maj" <fi...@adobe.com> wrote:
>I agree with Joe in that the specific, automated steps to create a
>sub-project's release won't help in streamlining the release process for
>this project. Appreciate your efforts to help in this regard though,
>Jukka.
>
>The underlying issue is that this project has a lot of shared resources.
>The javascript, example "hello world" app and the test suite are
>standalone projects but are used by all platform implementations. The crux
>of the slow release process is this: the shared projects need to be tagged
>before the platform implementations can be. If we find a problem in the
>shared projects, then all platforms need a retagging.
>
>If the above problem can be solved, then the release flow will improve.
>
>Perhaps this is more a matter of diligence and testing than a problem with
>the release steps themselves. If we are better at catching issues with
>shared sub-projects on all platforms, then the retag cycle (ideally) would
>be eliminated.
>
>Either way, there is ceremony and process involved. A lot of testing. Many
>devices. Lots of platforms. Communication and a bit of planning can help
>here. A week before a potential tag we could create an issue to track
>progress of testing JS+test suite+hello world sub-projects across all
>platforms. Only once all of those tasks are done can we with confidence
>say that the shared projects are stable and we can copy them into the
>platform implementations that we ship and tag those.
>
>Thoughts?
>
>On 9/20/12 11:32 AM, "Jukka Zitting" <ju...@gmail.com> wrote:
>
>>Hi,
>>
>>On Thu, Sep 20, 2012 at 6:45 PM, Joe Bowser <bo...@gmail.com> wrote:
>>> I know, I'm just saying that this has the same effect as releasing the
>>> components independently. I'm not convinced that this would reduce
>>> overhead when it comes to releases.
>>
>>OK, fair enough.
>>
>>> Also, how would this make us conform to the Apache requirements
>>> better?
>>
>>Not strict requirements as such (the current release package is IMHO
>>fine from a policy perspective), but rather a more pragmatic point of
>>view that underlies the Apache policies.
>>
>>The release guide [1] speaks of releases being "in the form of the
>>source materials needed to make changes to the software", which in
>>practice means that it's a good idea for the structure of the release
>>to be as close as possible to the structure of the source in the
>>version control system. The purpose of cutting a source release is not
>>to satisfy an abstract policy but rather to produce something that
>>downstream developers can easily use and modify to best match their
>>needs.
>>
>>When looking at the current 2.1.0 release candidate my first
>>impression is that there's no build script and the only instructions
>>on how to build this thing is to "change directories to the platform
>>that you wish to build for and read the README file." And to get to
>>those platform sources I first need to unpack them to a separate
>>directory. I might be wrong, but it seems to me that most people would
>>rather simply start from a tag of a relevant platfrom repository
>>instead of building the release candidate. If that assumption is
>>correct, then I think it would be a good idea for the release
>>structure to better match the expected access patterns.
>>
>>The main point I'm trying to address here is the grumbling I see about
>>the whole release process and its inefficiency. If the outcome of the
>>release process isn't something that's useful, then we're doing
>>something wrong and should fix that. If the process itself is too
>>complicated or otherwise takes too long, we should fix that too. I'd
>>love to hear also other ideas on how that could be done.
>>
>>[1] http://www.apache.org/dev/release.html#what
>>
>>BR,
>>
>>Jukka Zitting
>
Re: Lower-overhead release process
Posted by Filip Maj <fi...@adobe.com>.
I agree with Joe in that the specific, automated steps to create a
sub-project's release won't help in streamlining the release process for
this project. Appreciate your efforts to help in this regard though, Jukka.
The underlying issue is that this project has a lot of shared resources.
The javascript, example "hello world" app and the test suite are
standalone projects but are used by all platform implementations. The crux
of the slow release process is this: the shared projects need to be tagged
before the platform implementations can be. If we find a problem in the
shared projects, then all platforms need a retagging.
If the above problem can be solved, then the release flow will improve.
Perhaps this is more a matter of diligence and testing than a problem with
the release steps themselves. If we are better at catching issues with
shared sub-projects on all platforms, then the retag cycle (ideally) would
be eliminated.
Either way, there is ceremony and process involved. A lot of testing. Many
devices. Lots of platforms. Communication and a bit of planning can help
here. A week before a potential tag we could create an issue to track
progress of testing JS+test suite+hello world sub-projects across all
platforms. Only once all of those tasks are done can we with confidence
say that the shared projects are stable and we can copy them into the
platform implementations that we ship and tag those.
Thoughts?
On 9/20/12 11:32 AM, "Jukka Zitting" <ju...@gmail.com> wrote:
>Hi,
>
>On Thu, Sep 20, 2012 at 6:45 PM, Joe Bowser <bo...@gmail.com> wrote:
>> I know, I'm just saying that this has the same effect as releasing the
>> components independently. I'm not convinced that this would reduce
>> overhead when it comes to releases.
>
>OK, fair enough.
>
>> Also, how would this make us conform to the Apache requirements
>> better?
>
>Not strict requirements as such (the current release package is IMHO
>fine from a policy perspective), but rather a more pragmatic point of
>view that underlies the Apache policies.
>
>The release guide [1] speaks of releases being "in the form of the
>source materials needed to make changes to the software", which in
>practice means that it's a good idea for the structure of the release
>to be as close as possible to the structure of the source in the
>version control system. The purpose of cutting a source release is not
>to satisfy an abstract policy but rather to produce something that
>downstream developers can easily use and modify to best match their
>needs.
>
>When looking at the current 2.1.0 release candidate my first
>impression is that there's no build script and the only instructions
>on how to build this thing is to "change directories to the platform
>that you wish to build for and read the README file." And to get to
>those platform sources I first need to unpack them to a separate
>directory. I might be wrong, but it seems to me that most people would
>rather simply start from a tag of a relevant platfrom repository
>instead of building the release candidate. If that assumption is
>correct, then I think it would be a good idea for the release
>structure to better match the expected access patterns.
>
>The main point I'm trying to address here is the grumbling I see about
>the whole release process and its inefficiency. If the outcome of the
>release process isn't something that's useful, then we're doing
>something wrong and should fix that. If the process itself is too
>complicated or otherwise takes too long, we should fix that too. I'd
>love to hear also other ideas on how that could be done.
>
>[1] http://www.apache.org/dev/release.html#what
>
>BR,
>
>Jukka Zitting
Re: Lower-overhead release process
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Sep 20, 2012 at 6:45 PM, Joe Bowser <bo...@gmail.com> wrote:
> I know, I'm just saying that this has the same effect as releasing the
> components independently. I'm not convinced that this would reduce
> overhead when it comes to releases.
OK, fair enough.
> Also, how would this make us conform to the Apache requirements
> better?
Not strict requirements as such (the current release package is IMHO
fine from a policy perspective), but rather a more pragmatic point of
view that underlies the Apache policies.
The release guide [1] speaks of releases being "in the form of the
source materials needed to make changes to the software", which in
practice means that it's a good idea for the structure of the release
to be as close as possible to the structure of the source in the
version control system. The purpose of cutting a source release is not
to satisfy an abstract policy but rather to produce something that
downstream developers can easily use and modify to best match their
needs.
When looking at the current 2.1.0 release candidate my first
impression is that there's no build script and the only instructions
on how to build this thing is to "change directories to the platform
that you wish to build for and read the README file." And to get to
those platform sources I first need to unpack them to a separate
directory. I might be wrong, but it seems to me that most people would
rather simply start from a tag of a relevant platfrom repository
instead of building the release candidate. If that assumption is
correct, then I think it would be a good idea for the release
structure to better match the expected access patterns.
The main point I'm trying to address here is the grumbling I see about
the whole release process and its inefficiency. If the outcome of the
release process isn't something that's useful, then we're doing
something wrong and should fix that. If the process itself is too
complicated or otherwise takes too long, we should fix that too. I'd
love to hear also other ideas on how that could be done.
[1] http://www.apache.org/dev/release.html#what
BR,
Jukka Zitting
Re: Lower-overhead release process
Posted by Joe Bowser <bo...@gmail.com>.
I know, I'm just saying that this has the same effect as releasing the
components independently. I'm not convinced that this would reduce
overhead
when it comes to releases. Also, how would this make us conform to
the Apache requirements better?
On Thu, Sep 20, 2012 at 9:39 AM, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> 20.9.2012 18.02 "Joe Bowser" <bo...@gmail.com> kirjoitti:
>> I think this will make things worse since platform maintainers already are
>> out of sync, I can see release archives becoming completely disjointed
>> (Android on 2.1 while everything else is 2.0, for example). If we do this,
>> we may as well run 8 different projects instead of one.
>
> The idea is not to release the components independently, just to combine
> the packaging step with tagging. A Cordova release would still contain all
> the platforms, just as a collection of separate packages instead of one big
> archive.
>
> BR,
>
> Jukka Zitting
Re: Lower-overhead release process
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
20.9.2012 18.02 "Joe Bowser" <bo...@gmail.com> kirjoitti:
> I think this will make things worse since platform maintainers already are
> out of sync, I can see release archives becoming completely disjointed
> (Android on 2.1 while everything else is 2.0, for example). If we do this,
> we may as well run 8 different projects instead of one.
The idea is not to release the components independently, just to combine
the packaging step with tagging. A Cordova release would still contain all
the platforms, just as a collection of separate packages instead of one big
archive.
BR,
Jukka Zitting
Re: Lower-overhead release process
Posted by Joe Bowser <bo...@gmail.com>.
-1
I think this will make things worse since platform maintainers already are
out of sync, I can see release archives becoming completely disjointed
(Android on 2.1 while everything else is 2.0, for example). If we do this,
we may as well run 8 different projects instead of one.
I don't think this will solve our issues, instead it will make them worse.
On Sep 20, 2012 8:50 AM, "Jukka Zitting" <ju...@gmail.com> wrote:
> Hi,
>
> We've gone through a couple of iterations of the release process, but
> I still think that there's some room for improvement on better fitting
> the Apache requirements with the way the Cordova team manages
> releases. Here's one idea that I briefly discussed with Andre and Dave
> when they dropped by in Basel today.
>
> A distinctive feature of the Cordova release process is the tagging
> phase where people working on different platforms each tag the
> respective repositories. Currently these tags are then used by the
> release manager to put together the final release candidate. I think
> we could simplify this two-step process by asking everyone who tags a
> repository to also produce a release archive for that tag. With a
> cordova-release.sh script like the one I've included at the end of
> this message, this would be as simple as:
>
> $ cd incubator-cordova-android
> $ cordova-release.sh 2.2.0
>
> As a result you'll get a signed tag and a related signed and
> checksummed release tarball:
>
> $ git tag -v 2.2.0
> object 11dbf05c8452d045a25a59071201b963c52c8e83
> type commit
> tag 2.2.0
> tagger Jukka Zitting <ju...@apache.org> 1348155222 +0200
>
> incubator-cordova-android version 2.2.0
> gpg: Signature made Thu 20 Sep 2012 17:33:42 CEST using DSA key ID
> A355A63E
> gpg: Good signature from "Jukka Zitting <ju...@apache.org>"
>
> $ ls *.tar.gz*
> incubator-cordova-android-2.2.0.tar.gz
> incubator-cordova-android-2.2.0.tar.gz.asc
> incubator-cordova-android-2.2.0.tar.gz.md5
> incubator-cordova-android-2.2.0.tar.gz.sha
>
> You can then push the tag and upload the tarball to people.apache.org
> or wherever is most convenient. Also these steps could be automated.
>
> The responsibilities of the release manager are then to coordinate
> this effort, to collect all produced tarballs, to conduct the release
> vote once all the components are tagged, and finally to make the
> combined release available to the world.
>
> WDYT?
>
> BR,
>
> Jukka Zitting
>
> ----
> $ cat cordova-release.sh
> #!/bin/sh
>
> TAG="$1"
> if [ -z "$TAG" ]; then
> echo "usage: cordova-release.sh x.y.z" 1>&2
> exit 1
> fi
>
> DIR=`git config remote.origin.url | sed -e 's/.*\/\(.*\)\.git/\1/'`
> if [ -z "$DIR" ]; then
> echo "error: Unable to determine the repository name." 1>&2
> exit 2
> fi
>
> TGZ="$DIR-$TAG.tar.gz"
> if [ -e "$TGZ" ]; then
> echo "error: $TGZ already exists." 1>&2
> exit 3
> fi
>
> set -e
> git tag -s -m "$DIR version $TAG" "$TAG"
> git archive --output="$TGZ" --prefix="$DIR-$TAG/" "$TAG"
> gpg --armor --output "$TGZ.asc" --detach-sig "$TGZ"
> gpg --print-md MD5 "$TGZ" > "$TGZ.md5"
> gpg --print-md SHA512 "$TGZ" > "$TGZ.sha"
> exit 0
>