You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Otto Fowler <ot...@gmail.com> on 2018/06/26 17:21:40 UTC
[PROPOSAL] Reference Utility Script to help automate some of RC
validation in Release Process
I would like to propose the the Release Process
<http://nifi.apache.org/release-guide.html>, specifically the section:
*RM sends the following helper email to the NiFi Developers Mailing List*
be changed to include a link to, as well as a description of valid use of
the script referenced in PR 2806 <https://github.com/apache/nifi/pull/2806>.
This would be done as opposed to having the script in the NiFi source tree.
This script has been added to the NiFi wiki’s Releasing Nifi Page
<https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi> in the
scripts to help release nifi section.
The proposed change to the helper guide email:
Hello Apache NiFi community,
Please find the associated guidance to help those interested in
validating/verifying the release so they can vote.
# Download latest KEYS file:https://dist.apache.org/repos/dist/dev/nifi/KEYS
# Import keys file:
gpg --import KEYS
# [optional] Clear out local maven artifact repository
# Pull down nifi-${NIFI_VERSION} source release artifacts for review:
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.asc
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha1
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha256
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha512
# Verify the signature
gpg --verify nifi-${NIFI_VERSION}-source-release.zip.asc
# Verify the hashes (sha1, sha256, sha512) match the source and what
was provided in the vote email thread
shasum -a 1 nifi-${NIFI_VERSION}-source-release.zip
shasum -a 256 nifi-${NIFI_VERSION}-source-release.zip
shasum -a 512 nifi-${NIFI_VERSION}-source-release.zip
# Unzip nifi-${NIFI_VERSION}-source-release.zip
# Verify the build works including release audit tool (RAT) checks
cd nifi-${NIFI_VERSION}
mvn clean install -Pcontrib-check,include-grpc
# Verify the contents contain a good README, NOTICE, and LICENSE.
# Verify the git commit ID is correct
# Verify the RC was branched off the correct git commit ID
# Look at the resulting convenience binary as found in nifi-assembly/target
# Make sure the README, NOTICE, and LICENSE are present and correct
# Run the resulting convenience binary and make sure it works as expected
# Send a response to the vote thread indicating a +1, 0, -1 based on
your findings.
A script automating some of this process is available at
https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi as
release candidate check script.
While this script automates some of the steps above, it is not
replacement for all the checks above.
Thank you for your time and effort to validate the release!
Joe Witt’s original recommendation in PR 2806
<https://github.com/apache/nifi/pull/2806#issuecomment-400349540>.
Re: [PROPOSAL] Reference Utility Script to help automate some of RC
validation in Release Process
Posted by Peter Wilcsinszky <pe...@gmail.com>.
Hey Otto, I would like to try it and give feedback, I forgot to do this
last time.
On Mon, Aug 13, 2018 at 4:13 PM Otto Fowler <ot...@gmail.com> wrote:
> Does anyone have comment as to if they would be interested? I based the
> script on another committer’s personal script, and we do use a script like
> that
> with the Apache Metron project. I fully admit to the assumption that
> people would be interested in scripting some of this stuff away here too.
> If it is not going to be used
> however, I’d rather not clutter the wiki, and I’ll remove the script.
>
>
>
> On August 13, 2018 at 10:06:13, Joe Witt (joe.witt@gmail.com) wrote:
>
> Otto
>
> I think it is fine to update the email template that is used when
> sending the RC vote thread out to reference the script. What is
> problematic though is that it doesn't make clear which parts it does
> automate or which environments it is tested and will work in. What we
> dont want is for people to run that script and think they can or
> should skip various steps. You did well to highlight it doesn't do it
> all but i think being specific about only that which it does do is
> necessary.
>
> I personally don't find the steps outlined in the RC review process
> burdensome or worth automating so I will likely not be providing much
> input in that regard. But, for those that do want to leverage the
> script you'll want to be really explicit about what its usage covers
> precisely and in what environments it is tested and maintained.
>
> Thanks
> joe
> On Mon, Aug 13, 2018 at 9:57 AM Otto Fowler <ot...@gmail.com>
> wrote:
> >
> > Just re-upping this for next release.
> >
> >
> > On June 26, 2018 at 13:21:40, Otto Fowler (ottobackwards@gmail.com)
> wrote:
> >
> > I would like to propose the the Release Process
> > <http://nifi.apache.org/release-guide.html>, specifically the section:
> >
> > *RM sends the following helper email to the NiFi Developers Mailing List*
> >
> > be changed to include a link to, as well as a description of valid use of
> > the script referenced in PR 2806 <
> https://github.com/apache/nifi/pull/2806>.
>
> > This would be done as opposed to having the script in the NiFi source
> tree.
> >
> > This script has been added to the NiFi wiki’s Releasing Nifi Page
> > <https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi> in the
> > scripts to help release nifi section.
> >
> > The proposed change to the helper guide email:
> >
> > Hello Apache NiFi community,
> >
> > Please find the associated guidance to help those interested in
> > validating/verifying the release so they can vote.
> >
> > # Download latest KEYS file:
> https://dist.apache.org/repos/dist/dev/nifi/KEYS
> >
> > # Import keys file:
> > gpg --import KEYS
> >
> > # [optional] Clear out local maven artifact repository
> >
> > # Pull down nifi-${NIFI_VERSION} source release artifacts for review:
> >
> > wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip
> > wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.asc
> > wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha1
> > wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha256
> > wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha512
> >
> > # Verify the signature
> > gpg --verify nifi-${NIFI_VERSION}-source-release.zip.asc
> >
> > # Verify the hashes (sha1, sha256, sha512) match the source and what
> > was provided in the vote email thread
> > shasum -a 1 nifi-${NIFI_VERSION}-source-release.zip
> > shasum -a 256 nifi-${NIFI_VERSION}-source-release.zip
> > shasum -a 512 nifi-${NIFI_VERSION}-source-release.zip
> >
> > # Unzip nifi-${NIFI_VERSION}-source-release.zip
> >
> > # Verify the build works including release audit tool (RAT) checks
> > cd nifi-${NIFI_VERSION}
> > mvn clean install -Pcontrib-check,include-grpc
> >
> > # Verify the contents contain a good README, NOTICE, and LICENSE.
> >
> > # Verify the git commit ID is correct
> >
> > # Verify the RC was branched off the correct git commit ID
> >
> > # Look at the resulting convenience binary as found in
> nifi-assembly/target
> >
> > # Make sure the README, NOTICE, and LICENSE are present and correct
> >
> > # Run the resulting convenience binary and make sure it works as expected
> >
> > # Send a response to the vote thread indicating a +1, 0, -1 based on
> > your findings.
> >
> > A script automating some of this process is available at
> > https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi as
> > release candidate check script.
> > While this script automates some of the steps above, it is not
> > replacement for all the checks above.
> >
> >
> > Thank you for your time and effort to validate the release!
> >
> > Joe Witt’s original recommendation in PR 2806
> > <https://github.com/apache/nifi/pull/2806#issuecomment-400349540>.
>
Re: [PROPOSAL] Reference Utility Script to help automate some of RC
validation in Release Process
Posted by Otto Fowler <ot...@gmail.com>.
Does anyone have comment as to if they would be interested? I based the
script on another committer’s personal script, and we do use a script like
that
with the Apache Metron project. I fully admit to the assumption that
people would be interested in scripting some of this stuff away here too.
If it is not going to be used
however, I’d rather not clutter the wiki, and I’ll remove the script.
On August 13, 2018 at 10:06:13, Joe Witt (joe.witt@gmail.com) wrote:
Otto
I think it is fine to update the email template that is used when
sending the RC vote thread out to reference the script. What is
problematic though is that it doesn't make clear which parts it does
automate or which environments it is tested and will work in. What we
dont want is for people to run that script and think they can or
should skip various steps. You did well to highlight it doesn't do it
all but i think being specific about only that which it does do is
necessary.
I personally don't find the steps outlined in the RC review process
burdensome or worth automating so I will likely not be providing much
input in that regard. But, for those that do want to leverage the
script you'll want to be really explicit about what its usage covers
precisely and in what environments it is tested and maintained.
Thanks
joe
On Mon, Aug 13, 2018 at 9:57 AM Otto Fowler <ot...@gmail.com>
wrote:
>
> Just re-upping this for next release.
>
>
> On June 26, 2018 at 13:21:40, Otto Fowler (ottobackwards@gmail.com)
wrote:
>
> I would like to propose the the Release Process
> <http://nifi.apache.org/release-guide.html>, specifically the section:
>
> *RM sends the following helper email to the NiFi Developers Mailing List*
>
> be changed to include a link to, as well as a description of valid use of
> the script referenced in PR 2806 <https://github.com/apache/nifi/pull/2806>.
> This would be done as opposed to having the script in the NiFi source
tree.
>
> This script has been added to the NiFi wiki’s Releasing Nifi Page
> <https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi> in the
> scripts to help release nifi section.
>
> The proposed change to the helper guide email:
>
> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
> gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down nifi-${NIFI_VERSION} source release artifacts for review:
>
> wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip
> wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.asc
> wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha1
> wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha256
> wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha512
>
> # Verify the signature
> gpg --verify nifi-${NIFI_VERSION}-source-release.zip.asc
>
> # Verify the hashes (sha1, sha256, sha512) match the source and what
> was provided in the vote email thread
> shasum -a 1 nifi-${NIFI_VERSION}-source-release.zip
> shasum -a 256 nifi-${NIFI_VERSION}-source-release.zip
> shasum -a 512 nifi-${NIFI_VERSION}-source-release.zip
>
> # Unzip nifi-${NIFI_VERSION}-source-release.zip
>
> # Verify the build works including release audit tool (RAT) checks
> cd nifi-${NIFI_VERSION}
> mvn clean install -Pcontrib-check,include-grpc
>
> # Verify the contents contain a good README, NOTICE, and LICENSE.
>
> # Verify the git commit ID is correct
>
> # Verify the RC was branched off the correct git commit ID
>
> # Look at the resulting convenience binary as found in
nifi-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
> # Send a response to the vote thread indicating a +1, 0, -1 based on
> your findings.
>
> A script automating some of this process is available at
> https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi as
> release candidate check script.
> While this script automates some of the steps above, it is not
> replacement for all the checks above.
>
>
> Thank you for your time and effort to validate the release!
>
> Joe Witt’s original recommendation in PR 2806
> <https://github.com/apache/nifi/pull/2806#issuecomment-400349540>.
Re: [PROPOSAL] Reference Utility Script to help automate some of RC
validation in Release Process
Posted by Joe Witt <jo...@gmail.com>.
Otto
I think it is fine to update the email template that is used when
sending the RC vote thread out to reference the script. What is
problematic though is that it doesn't make clear which parts it does
automate or which environments it is tested and will work in. What we
dont want is for people to run that script and think they can or
should skip various steps. You did well to highlight it doesn't do it
all but i think being specific about only that which it does do is
necessary.
I personally don't find the steps outlined in the RC review process
burdensome or worth automating so I will likely not be providing much
input in that regard. But, for those that do want to leverage the
script you'll want to be really explicit about what its usage covers
precisely and in what environments it is tested and maintained.
Thanks
joe
On Mon, Aug 13, 2018 at 9:57 AM Otto Fowler <ot...@gmail.com> wrote:
>
> Just re-upping this for next release.
>
>
> On June 26, 2018 at 13:21:40, Otto Fowler (ottobackwards@gmail.com) wrote:
>
> I would like to propose the the Release Process
> <http://nifi.apache.org/release-guide.html>, specifically the section:
>
> *RM sends the following helper email to the NiFi Developers Mailing List*
>
> be changed to include a link to, as well as a description of valid use of
> the script referenced in PR 2806 <https://github.com/apache/nifi/pull/2806>.
> This would be done as opposed to having the script in the NiFi source tree.
>
> This script has been added to the NiFi wiki’s Releasing Nifi Page
> <https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi> in the
> scripts to help release nifi section.
>
> The proposed change to the helper guide email:
>
> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
> gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down nifi-${NIFI_VERSION} source release artifacts for review:
>
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.asc
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha1
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha256
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha512
>
> # Verify the signature
> gpg --verify nifi-${NIFI_VERSION}-source-release.zip.asc
>
> # Verify the hashes (sha1, sha256, sha512) match the source and what
> was provided in the vote email thread
> shasum -a 1 nifi-${NIFI_VERSION}-source-release.zip
> shasum -a 256 nifi-${NIFI_VERSION}-source-release.zip
> shasum -a 512 nifi-${NIFI_VERSION}-source-release.zip
>
> # Unzip nifi-${NIFI_VERSION}-source-release.zip
>
> # Verify the build works including release audit tool (RAT) checks
> cd nifi-${NIFI_VERSION}
> mvn clean install -Pcontrib-check,include-grpc
>
> # Verify the contents contain a good README, NOTICE, and LICENSE.
>
> # Verify the git commit ID is correct
>
> # Verify the RC was branched off the correct git commit ID
>
> # Look at the resulting convenience binary as found in nifi-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
> # Send a response to the vote thread indicating a +1, 0, -1 based on
> your findings.
>
> A script automating some of this process is available at
> https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi as
> release candidate check script.
> While this script automates some of the steps above, it is not
> replacement for all the checks above.
>
>
> Thank you for your time and effort to validate the release!
>
> Joe Witt’s original recommendation in PR 2806
> <https://github.com/apache/nifi/pull/2806#issuecomment-400349540>.
Re: [PROPOSAL] Reference Utility Script to help automate some of RC
validation in Release Process
Posted by Otto Fowler <ot...@gmail.com>.
Just re-upping this for next release.
On June 26, 2018 at 13:21:40, Otto Fowler (ottobackwards@gmail.com) wrote:
I would like to propose the the Release Process
<http://nifi.apache.org/release-guide.html>, specifically the section:
*RM sends the following helper email to the NiFi Developers Mailing List*
be changed to include a link to, as well as a description of valid use of
the script referenced in PR 2806 <https://github.com/apache/nifi/pull/2806>.
This would be done as opposed to having the script in the NiFi source tree.
This script has been added to the NiFi wiki’s Releasing Nifi Page
<https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi> in the
scripts to help release nifi section.
The proposed change to the helper guide email:
Hello Apache NiFi community,
Please find the associated guidance to help those interested in
validating/verifying the release so they can vote.
# Download latest KEYS file:https://dist.apache.org/repos/dist/dev/nifi/KEYS
# Import keys file:
gpg --import KEYS
# [optional] Clear out local maven artifact repository
# Pull down nifi-${NIFI_VERSION} source release artifacts for review:
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.asc
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha1
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha256
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip.sha512
# Verify the signature
gpg --verify nifi-${NIFI_VERSION}-source-release.zip.asc
# Verify the hashes (sha1, sha256, sha512) match the source and what
was provided in the vote email thread
shasum -a 1 nifi-${NIFI_VERSION}-source-release.zip
shasum -a 256 nifi-${NIFI_VERSION}-source-release.zip
shasum -a 512 nifi-${NIFI_VERSION}-source-release.zip
# Unzip nifi-${NIFI_VERSION}-source-release.zip
# Verify the build works including release audit tool (RAT) checks
cd nifi-${NIFI_VERSION}
mvn clean install -Pcontrib-check,include-grpc
# Verify the contents contain a good README, NOTICE, and LICENSE.
# Verify the git commit ID is correct
# Verify the RC was branched off the correct git commit ID
# Look at the resulting convenience binary as found in nifi-assembly/target
# Make sure the README, NOTICE, and LICENSE are present and correct
# Run the resulting convenience binary and make sure it works as expected
# Send a response to the vote thread indicating a +1, 0, -1 based on
your findings.
A script automating some of this process is available at
https://cwiki.apache.org/confluence/display/NIFI/Releasing+NiFi as
release candidate check script.
While this script automates some of the steps above, it is not
replacement for all the checks above.
Thank you for your time and effort to validate the release!
Joe Witt’s original recommendation in PR 2806
<https://github.com/apache/nifi/pull/2806#issuecomment-400349540>.