You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Francis Chuang <fr...@apache.org> on 2018/06/25 23:47:33 UTC
Improving the developer experience for releasing
I'd like to open up some discussion regarding the developer experience
for making releases.
The maven configuration shipped with the project already does a lot of
heavy lifting and automation, but I ran into a few issues as a non-Java
developer.
Here are some of the things I ran into, while releasing Avatica 1.12.
1. GPG signs using a default key (I think this is the first key, if no
default is set). I had multiple keys and my Apache key was not my first key.
2. I was not 100% sure that `-DdevelopmentVersion` should be the version
after the current release.
3. It took a while to work out how to authenticate against Apache's
maven repo.
4. I did not have a suitable environment. OpenJDK did not install in WSL
correctly, while Oracle JDK did, but I think all releases should be
built using OpenJDK. I ended up spinning up an OpenJDK container using
the maven:alpine docker image and using it to execute the commands for
building.
For 1, I think it should be possible to pass in the key to be used in
`-Darguments` (will need to experiment).
For 2, I will update the documentation.
For 3, I will also update the documentation and reference
http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env
For 4, I would like to propose adding docker-compose.yaml with a bash
script that will automate the different steps for executing the dry run,
publishing or the RC and the final build promotion in a docker image.
This would be extremely useful for releases to easily build and make
releases (with safety checks in place) to reduce the time needed to make
releases drastically. This does not replace the current "more manual"
method of making releases, but offers a second option for those who are
not primary java developers.
What do you think?
Francis
Re: Improving the developer experience for releasing
Posted by Francis Chuang <fr...@apache.org>.
I've filed a JIRA and will be updating the HOWTO.
I also encountered an issue when performing a dry run of a release for
avatica
(https://calcite.apache.org/avatica/docs/howto.html#making-a-release-for-calcite-committers)
that I hope someone can answer.
The HOWTO states that we should run a script to update the Dockerfile's
AVATICA_VERSION to include `-SNAPSHOT`, for example `1.12.0-SNAPSHOT`.
The problem was that:
- If I ran the script and then executed the dry run, maven will fail due
to there being a modified file in the repository.
- If I did not run the script, maven will fail, because it expects
AVATICA_VERSION in the docker file to be `1.12.0-SNAPSHOT` and not `1.12.0`.
I think we can update pom.xml to include the following for the maven
release plugin, but would like some confirmation:
<configuration>
<checkModificationExcludes>
<checkModificationExclude>docker/src/main/dockerhub/Dockerfile</checkModificationExclude>
</checkModificationExcludes>
</configuration>
Francis
On 26/06/2018 10:05 AM, Julian Hyde wrote:
> Thanks for starting off this discussion. I presume you’re mainly thinking of the Avatica process but all except 4 also apply to Calcite.
>
> I’ve not personally run into 1, 2 or 3 but I can see how other people might. Feel free to update HOWTO if that would solve the problem. (Blame me for 2, especially. I introduced the “X.Y.Z” and “X.Y+1.Z-SNAPSHOT” notation, and it’s not helpful.)
>
> I don’t have an opinion about 4, because I’m not an expert on Docker and I’d never heard of WSL.
>
> Julian
>
>
>> On Jun 25, 2018, at 4:47 PM, Francis Chuang <fr...@apache.org> wrote:
>>
>> I'd like to open up some discussion regarding the developer experience for making releases.
>>
>> The maven configuration shipped with the project already does a lot of heavy lifting and automation, but I ran into a few issues as a non-Java developer.
>>
>> Here are some of the things I ran into, while releasing Avatica 1.12.
>> 1. GPG signs using a default key (I think this is the first key, if no default is set). I had multiple keys and my Apache key was not my first key.
>> 2. I was not 100% sure that `-DdevelopmentVersion` should be the version after the current release.
>> 3. It took a while to work out how to authenticate against Apache's maven repo.
>> 4. I did not have a suitable environment. OpenJDK did not install in WSL correctly, while Oracle JDK did, but I think all releases should be built using OpenJDK. I ended up spinning up an OpenJDK container using the maven:alpine docker image and using it to execute the commands for building.
>>
>> For 1, I think it should be possible to pass in the key to be used in `-Darguments` (will need to experiment).
>> For 2, I will update the documentation.
>> For 3, I will also update the documentation and reference http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env
>> For 4, I would like to propose adding docker-compose.yaml with a bash script that will automate the different steps for executing the dry run, publishing or the RC and the final build promotion in a docker image. This would be extremely useful for releases to easily build and make releases (with safety checks in place) to reduce the time needed to make releases drastically. This does not replace the current "more manual" method of making releases, but offers a second option for those who are not primary java developers.
>>
>> What do you think?
>>
>> Francis
Re: Improving the developer experience for releasing
Posted by Julian Hyde <jh...@apache.org>.
Thanks for starting off this discussion. I presume you’re mainly thinking of the Avatica process but all except 4 also apply to Calcite.
I’ve not personally run into 1, 2 or 3 but I can see how other people might. Feel free to update HOWTO if that would solve the problem. (Blame me for 2, especially. I introduced the “X.Y.Z” and “X.Y+1.Z-SNAPSHOT” notation, and it’s not helpful.)
I don’t have an opinion about 4, because I’m not an expert on Docker and I’d never heard of WSL.
Julian
> On Jun 25, 2018, at 4:47 PM, Francis Chuang <fr...@apache.org> wrote:
>
> I'd like to open up some discussion regarding the developer experience for making releases.
>
> The maven configuration shipped with the project already does a lot of heavy lifting and automation, but I ran into a few issues as a non-Java developer.
>
> Here are some of the things I ran into, while releasing Avatica 1.12.
> 1. GPG signs using a default key (I think this is the first key, if no default is set). I had multiple keys and my Apache key was not my first key.
> 2. I was not 100% sure that `-DdevelopmentVersion` should be the version after the current release.
> 3. It took a while to work out how to authenticate against Apache's maven repo.
> 4. I did not have a suitable environment. OpenJDK did not install in WSL correctly, while Oracle JDK did, but I think all releases should be built using OpenJDK. I ended up spinning up an OpenJDK container using the maven:alpine docker image and using it to execute the commands for building.
>
> For 1, I think it should be possible to pass in the key to be used in `-Darguments` (will need to experiment).
> For 2, I will update the documentation.
> For 3, I will also update the documentation and reference http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env
> For 4, I would like to propose adding docker-compose.yaml with a bash script that will automate the different steps for executing the dry run, publishing or the RC and the final build promotion in a docker image. This would be extremely useful for releases to easily build and make releases (with safety checks in place) to reduce the time needed to make releases drastically. This does not replace the current "more manual" method of making releases, but offers a second option for those who are not primary java developers.
>
> What do you think?
>
> Francis
Re: Improving the developer experience for releasing
Posted by Julian Hyde <jh...@apache.org>.
By the way, thank you for being release manager, Francis. It is of the fastest and most complex Avatica release I can remember. Thank you for making it run so smoothly.
Julian
> On Jun 25, 2018, at 4:47 PM, Francis Chuang <fr...@apache.org> wrote:
>
> I'd like to open up some discussion regarding the developer experience for making releases.
>
> The maven configuration shipped with the project already does a lot of heavy lifting and automation, but I ran into a few issues as a non-Java developer.
>
> Here are some of the things I ran into, while releasing Avatica 1.12.
> 1. GPG signs using a default key (I think this is the first key, if no default is set). I had multiple keys and my Apache key was not my first key.
> 2. I was not 100% sure that `-DdevelopmentVersion` should be the version after the current release.
> 3. It took a while to work out how to authenticate against Apache's maven repo.
> 4. I did not have a suitable environment. OpenJDK did not install in WSL correctly, while Oracle JDK did, but I think all releases should be built using OpenJDK. I ended up spinning up an OpenJDK container using the maven:alpine docker image and using it to execute the commands for building.
>
> For 1, I think it should be possible to pass in the key to be used in `-Darguments` (will need to experiment).
> For 2, I will update the documentation.
> For 3, I will also update the documentation and reference http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env
> For 4, I would like to propose adding docker-compose.yaml with a bash script that will automate the different steps for executing the dry run, publishing or the RC and the final build promotion in a docker image. This would be extremely useful for releases to easily build and make releases (with safety checks in place) to reduce the time needed to make releases drastically. This does not replace the current "more manual" method of making releases, but offers a second option for those who are not primary java developers.
>
> What do you think?
>
> Francis