You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@royale.apache.org by Alex Harui <ah...@adobe.com.INVALID> on 2018/12/05 18:54:30 UTC
[DISCUSS] Release Automation (was Re: svn commit: r31364 -
/dev/royale/0.9.4/rc2/ /release/royale/0.9.4/)
Hi Dave,
Feel free to make a proposal.
I'm not trying to kill any discussion, but I know for me personally, I would much rather debug an Ant script than debug English instructions, and consider that not all of our committers are native English speakers. I am really bad at following a long list of English instructions and am likely to make small errors that matter, like mis-typing versions. Right now, I can type the release version in once and rely on command prompt history to get it right after that.
It seems wrong that a project like Royale, which produces an SDK for desktop applications, would have a lot of manual steps in creating a release. In fact, Flex has an Ant-on-AIR library, such that Royale could write an AIR app to generate a release, similar to the Flex Installer. Then Royale RMs would be debugging ActionScript to add fault tolerance to the release process. Maybe that would make the process more accessible.
The other topic area I've wondered about is whether we could generate our releases on builds.a.o (or my CI server). I don't think the Jenkins nodes are allowed to push to Git or SVN and I think Maven' Release Plugin requires that. We could put my private key on my CI server, but I'd rather it be clear which commits came from the server and not actually me. But if there was a machine that built the release artifacts, RM's wouldn't have to configure their machines or deal with network issues. I've been intrigued by the discussion about reproducible binaries on other Apache lists. If we made Royale binaries reproducible, then it could be possible to ship binaries from a release server since the PMC voters could build the source packages and it would match the binaries from that server. AIUI, RM don't have to build the artifacts on their computers, they only have to verify the artifacts.
Yet another topic area is: who is going to do any work to change how we cut releases? It seems like other PMC members do not have the time to truly understand the process or to try to improve on it.
My 2 cents,
-Alex
On 12/4/18, 11:05 PM, "Dave Fisher" <da...@comcast.net> wrote:
Alex,
Thanks for taking care of this. Can we start a discussion about which parts of the release should be “manual” visa vi fully automated.
Everyone - let’s try to follow KISS - keep it stupid / simple.
Regards,
Dave
Sent from my iPhone
> On Dec 4, 2018, at 10:44 PM, aharui@apache.org wrote:
>
> Author: aharui
> Date: Wed Dec 5 06:44:34 2018
> New Revision: 31364
>
> Log:
> Apache Royale 0.9.4
>
> Added:
> release/royale/0.9.4/
> - copied from r31363, dev/royale/0.9.4/rc2/
> Removed:
> dev/royale/0.9.4/rc2/
>
Re: [DISCUSS] Release Automation (was Re: svn commit: r31364 -
/dev/royale/0.9.4/rc2/ /release/royale/0.9.4/)
Posted by Alex Harui <ah...@adobe.com.INVALID>.
Hi Dave,
There is a single Ant script called releasecandidate.xml, but it has multiple targets in it. 3 targets are used in your step 1, although I don't understand what you mean by staging. I think Maven requires staging for voters to test the bits.
A separate target is used for 3, although we cannot currently copy to dist or npm from repository.a.o since only Maven artifacts are on repository.a.o and we push a non-Maven artifact to npm and dist.
I'm pretty sure I understand the Apache policy and the scripts are following it, and I used them as RM for 0.9.4 correctly. Please be more specific if you wish to drill down on some aspect of it.
I agree that creating packages should be simple, but so far, nobody but me has been able to get the Maven commands to complete without errors that all seem to be related to network issues, not issues in the tools or scripts. It is possible that we could change our packaging, but has its own drawbacks. We want to use Maven to test that Maven support will work and we want to use Ant to prove our Ant support works. I'm told there are ways to change the Maven commands to try to upload all of the artifacts after they are built instead of as they are being built, but I personally don't want to take the time to try it out, since I might end up doing all of that work and it won't help folks with unreliable networks. That's why I thought that if we could build releasable artifacts on builds.a.o, we could trust its network reliability to repository.a.o.
Regarding reproducible builds, there are some clues that it is or will be possible soon:
https://dzone.com/articles/reproducible-builds-in-java
http://zlika.github.io/reproducible-build-maven-plugin/
We would have to modify the SWF compiler as well. But it could be worth doing if it helps make it easier to cut releases.
-Alex
On 12/7/18, 11:56 AM, "Dave Fisher" <da...@comcast.net> wrote:
Hi - Alex,
Inline.
> On Dec 5, 2018, at 10:54 AM, Alex Harui <ah...@adobe.com.INVALID> wrote:
>
> Hi Dave,
>
> Feel free to make a proposal.
I think that it is important to clearly separate the following.
(1) Preparing Release Packaging. Source release and the several binary packages optionally pushed to dist.apache.org/dev but NOT staged. Staging seems to get us in trouble.
(2) VOTE on the Release Candidate.
(3) Distribute the Release. Copying to dist, npm and maven through repository.apache.org. Updating our download pages. Waiting for Apache Mirror distribution. Sending the Announcement Email.
(1) and (3) must be distinctly separate processes.
(1) can be mvn and ant. Anyone should be able to do it whether or not they are release manager
(3) can be a shell script, mvn, and/or ant. Sending the announcement can be manual.
>
> I'm not trying to kill any discussion, but I know for me personally, I would much rather debug an Ant script than debug English instructions, and consider that not all of our committers are native English speakers. I am really bad at following a long list of English instructions and am likely to make small errors that matter, like mis-typing versions. Right now, I can type the release version in once and rely on command prompt history to get it right after that.
>
> It seems wrong that a project like Royale, which produces an SDK for desktop applications, would have a lot of manual steps in creating a release. In fact, Flex has an Ant-on-AIR library, such that Royale could write an AIR app to generate a release, similar to the Flex Installer. Then Royale RMs would be debugging ActionScript to add fault tolerance to the release process. Maybe that would make the process more accessible.
Creating packages should be very simple. Agreed. In the Apache Way the release of those packages requires a VOTE of at least 72 hours (unless there are very good reasons to be faster.) You cannot really have one script to both create the packages and to release them.
>
> The other topic area I've wondered about is whether we could generate our releases on builds.a.o (or my CI server). I don't think the Jenkins nodes are allowed to push to Git or SVN and I think Maven' Release Plugin requires that.
This is a question for builds@apache.org and/or try some things on GitHub with TravisCI. This would be for creating packages.
> We could put my private key on my CI server, but I'd rather it be clear which commits came from the server and not actually me. But if there was a machine that built the release artifacts, RM's wouldn't have to configure their machines or deal with network issues. I've been intrigued by the discussion about reproducible binaries on other Apache lists. If we made Royale binaries reproducible, then it could be possible to ship binaries from a release server since the PMC voters could build the source packages and it would match the binaries from that server. AIUI, RM don't have to build the artifacts on their computers, they only have to verify the artifacts.
JAR files are never exactly the same. There are date time stamps in there along with slightly different byte code depending on exactly which Java SDK is used.
>
> Yet another topic area is: who is going to do any work to change how we cut releases? It seems like other PMC members do not have the time to truly understand the process or to try to improve on it.
My real point is that I think that the (1), (2), (3) separation required by the Apache Way has never been properly followed, but I could be wrong.
Perhaps I’ll have time to look later this month. No promises.
Regards,
Dave
>
> My 2 cents,
> -Alex
>
> On 12/4/18, 11:05 PM, "Dave Fisher" <da...@comcast.net> wrote:
>
> Alex,
>
> Thanks for taking care of this. Can we start a discussion about which parts of the release should be “manual” visa vi fully automated.
>
> Everyone - let’s try to follow KISS - keep it stupid / simple.
>
> Regards,
> Dave
>
> Sent from my iPhone
>
>> On Dec 4, 2018, at 10:44 PM, aharui@apache.org wrote:
>>
>> Author: aharui
>> Date: Wed Dec 5 06:44:34 2018
>> New Revision: 31364
>>
>> Log:
>> Apache Royale 0.9.4
>>
>> Added:
>> release/royale/0.9.4/
>> - copied from r31363, dev/royale/0.9.4/rc2/
>> Removed:
>> dev/royale/0.9.4/rc2/
>>
>
>
>
Re: [DISCUSS] Release Automation (was Re: svn commit: r31364 -
/dev/royale/0.9.4/rc2/ /release/royale/0.9.4/)
Posted by Dave Fisher <da...@comcast.net>.
Hi - Alex,
Inline.
> On Dec 5, 2018, at 10:54 AM, Alex Harui <ah...@adobe.com.INVALID> wrote:
>
> Hi Dave,
>
> Feel free to make a proposal.
I think that it is important to clearly separate the following.
(1) Preparing Release Packaging. Source release and the several binary packages optionally pushed to dist.apache.org/dev but NOT staged. Staging seems to get us in trouble.
(2) VOTE on the Release Candidate.
(3) Distribute the Release. Copying to dist, npm and maven through repository.apache.org. Updating our download pages. Waiting for Apache Mirror distribution. Sending the Announcement Email.
(1) and (3) must be distinctly separate processes.
(1) can be mvn and ant. Anyone should be able to do it whether or not they are release manager
(3) can be a shell script, mvn, and/or ant. Sending the announcement can be manual.
>
> I'm not trying to kill any discussion, but I know for me personally, I would much rather debug an Ant script than debug English instructions, and consider that not all of our committers are native English speakers. I am really bad at following a long list of English instructions and am likely to make small errors that matter, like mis-typing versions. Right now, I can type the release version in once and rely on command prompt history to get it right after that.
>
> It seems wrong that a project like Royale, which produces an SDK for desktop applications, would have a lot of manual steps in creating a release. In fact, Flex has an Ant-on-AIR library, such that Royale could write an AIR app to generate a release, similar to the Flex Installer. Then Royale RMs would be debugging ActionScript to add fault tolerance to the release process. Maybe that would make the process more accessible.
Creating packages should be very simple. Agreed. In the Apache Way the release of those packages requires a VOTE of at least 72 hours (unless there are very good reasons to be faster.) You cannot really have one script to both create the packages and to release them.
>
> The other topic area I've wondered about is whether we could generate our releases on builds.a.o (or my CI server). I don't think the Jenkins nodes are allowed to push to Git or SVN and I think Maven' Release Plugin requires that.
This is a question for builds@apache.org and/or try some things on GitHub with TravisCI. This would be for creating packages.
> We could put my private key on my CI server, but I'd rather it be clear which commits came from the server and not actually me. But if there was a machine that built the release artifacts, RM's wouldn't have to configure their machines or deal with network issues. I've been intrigued by the discussion about reproducible binaries on other Apache lists. If we made Royale binaries reproducible, then it could be possible to ship binaries from a release server since the PMC voters could build the source packages and it would match the binaries from that server. AIUI, RM don't have to build the artifacts on their computers, they only have to verify the artifacts.
JAR files are never exactly the same. There are date time stamps in there along with slightly different byte code depending on exactly which Java SDK is used.
>
> Yet another topic area is: who is going to do any work to change how we cut releases? It seems like other PMC members do not have the time to truly understand the process or to try to improve on it.
My real point is that I think that the (1), (2), (3) separation required by the Apache Way has never been properly followed, but I could be wrong.
Perhaps I’ll have time to look later this month. No promises.
Regards,
Dave
>
> My 2 cents,
> -Alex
>
> On 12/4/18, 11:05 PM, "Dave Fisher" <da...@comcast.net> wrote:
>
> Alex,
>
> Thanks for taking care of this. Can we start a discussion about which parts of the release should be “manual” visa vi fully automated.
>
> Everyone - let’s try to follow KISS - keep it stupid / simple.
>
> Regards,
> Dave
>
> Sent from my iPhone
>
>> On Dec 4, 2018, at 10:44 PM, aharui@apache.org wrote:
>>
>> Author: aharui
>> Date: Wed Dec 5 06:44:34 2018
>> New Revision: 31364
>>
>> Log:
>> Apache Royale 0.9.4
>>
>> Added:
>> release/royale/0.9.4/
>> - copied from r31363, dev/royale/0.9.4/rc2/
>> Removed:
>> dev/royale/0.9.4/rc2/
>>
>
>
>