You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by ottobackwards <gi...@git.apache.org> on 2018/06/20 19:01:47 UTC
[GitHub] nifi pull request #2806: NIFI-5068 [WIP] Script to automate RC validation
GitHub user ottobackwards opened a pull request:
https://github.com/apache/nifi/pull/2806
NIFI-5068 [WIP] Script to automate RC validation
This is a work in process.
Currently this script will download and validate the hashes and signing, as well as unpack the zip.
Still need to:
- validate the commit
- run tests
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/nifi rc-check
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2806.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2806
----
commit d7b41852f04d6d1d7027f95b11bf1eafb8d52282
Author: Otto Fowler <ot...@...>
Date: 2018-04-11T14:39:59Z
start of utility, though there is no download to test against
commit d0cd56cdba5ad3c8bf668068883a40193e51ffdc
Author: Otto Fowler <ot...@...>
Date: 2018-06-11T15:39:58Z
Merge branch 'master' of https://github.com/apache/nifi into rc-check
commit efad5da1557fba3f88bb3a8c45d64d523b674924
Author: Otto Fowler <ot...@...>
Date: 2018-06-14T19:58:51Z
some rework
commit 41995d5c57405cf10fe17d242dd75a1014b92f31
Author: Otto Fowler <ot...@...>
Date: 2018-06-14T20:19:04Z
works up until this point, but only with a release, need to have switch
commit 5081c1536d29b956abb1245a47a44629f9f784ea
Author: Otto Fowler <ot...@...>
Date: 2018-06-20T18:59:33Z
working for 1.7.0 rc
----
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
closing, not much demand I guess.
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by joewitt <gi...@git.apache.org>.
Github user joewitt commented on the issue:
https://github.com/apache/nifi/pull/2806
should have them now
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
How would we go about getting it on the wiki page?
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
ok @joewitt, but did it work? ;)
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate RC validation
Posted by joewitt <gi...@git.apache.org>.
Github user joewitt commented on the issue:
https://github.com/apache/nifi/pull/2806
i dont think we should commit this into the codebase. It could be fine to publish the bash script in the wiki page associated with release guide/validation. We must also be clear this is good for things which are easily automated such as verification of sigs/hashes/build but for other things like actually testing nifi and the hard work of L&N review its an eyes-on game.
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
@joewitt I agree. I'll see if I can edit the wiki ( if I have rights )
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
I do not have rights to create a page in the space from what I can tell
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
I sent the email, I'll close this PR with the results of that when they are set
---
[GitHub] nifi pull request #2806: NIFI-5068 Script to automate parts of RC validation
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2806#discussion_r198000993
--- Diff: dev-utilities/release-utilities/nifi-rc-check ---
@@ -0,0 +1,326 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+shopt -s nocasematch
+
+function print_help {
+ echo " "
+ echo "usage: ${0}"
+ echo " -v/--version=<version> The version of the Nifi release. [Required]"
+ echo " -c/--candidate=<RC#> Defines the Release Candidate. [Required]"
+ echo " -h/--help Usage information."
+ echo " "
+ echo "example: "
+ echo " nifi-rc-check --version=1.6.0 --candidate=RC2"
+ echo " "
+}
+
+print_section_header() {
+ echo
+ echo ----------------------------------------------
+ echo
+ echo "$@"
+ echo
+ echo ----------------------------------------------
+ echo
+}
+
+print_section_item() {
+ echo
+ echo "-----> $@"
+ echo
+}
+
+print_item() {
+ echo
+ echo "$@"
+ echo
+}
+
+print_error() {
+ echo
+ echo "ERROR !!!!!!"
+ echo ----------------------------------------------
+ echo
+ echo "$@"
+ echo
+ echo ----------------------------------------------
+ echo
+}
+
+NIFI_DEV_DIST="https://dist.apache.org/repos/dist/dev/nifi/"
+NIFI_DIST="https://www.apache.org/dist/nifi/"
+NIFI_NAME_LOWER="nifi";
+
+# print help, if the user just runs this without any args
+if [ "$#" -eq 0 ]; then
+ print_help
+ exit 1
+fi
+
+# handle command line options
+for i in "$@"; do
+ case ${i} in
+ #
+ # VERSION: The release version of Nifi to validate.
+ #
+ #
+ -v=*|--version=*)
+ VERSION="${i#*=}"
+ shift # past argument=value
+ ;;
+
+ #
+ # RC: Defines the RC# to use
+ #
+ # -c=RC2
+ # --candidate=RC2
+ #
+ -c=*|--candidate=*)
+ CANDIDATE="${i#*=}"
+ shift # past argument=value
+ ;;
+
+ #
+ # -h/--help
+ #
+ -h|--help)
+ help
+ exit 0
+ shift # past argument with no value
+ ;;
+
+ #
+ # Unknown option
+ #
+ *)
+ UNKNOWN_OPTION="${i#*=}"
+ echo "Error: unknown option: $UNKNOWN_OPTION"
+ help
+ ;;
+ esac
+done
+
+# validation
+if [ -z "$VERSION" ]; then
+ echo "Missing -v/--version is is required"
+ exit 1
+fi
+if [[ "$VERSION" =~ ^[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2} ]]; then
+ NIFI_VERSION="$VERSION"
+else
+ print_error "$VERSION may not be a valid version number"
+ exit 1
+fi
+
+if [ -z "$CANDIDATE" ]; then
+ echo "Missing -c/--candidate which is required"
+ exit 1
+fi
+
+if [[ "$CANDIDATE" =~ ^RC[0-9]+ ]]; then
+ RC=$(echo "$CANDIDATE" | tr '[:upper:]' '[:lower:]')
+ UPPER_RC=$(echo "$CANDIDATE" | tr '[:lower:]' '[:upper:]')
+elif [[ "$CANDIDATE" =~ ^[0-9]+ ]]; then
+ RC=rc"$CANDIDATE"
+ UPPER_RC=RC"$CANDIDATE"
+else
+ print_error "invalid RC, valid is RC# or just #"
+ exit 1
+fi
+
+print_section_header "Apache Nifi Version $NIFI_VERSION $UPPER_RC"
+
+NIFI_REL_RC="$NIFI_NAME_LOWER-$NIFI_VERSION-$UPPER_RC"
+
+NIFI_RC_DIST="$NIFI_DEV_DIST$NIFI_NAME_LOWER-$NIFI_VERSION/"
+print_item "Apache Nifi RC Distribution Root is $NIFI_RC_DIST"
+
+
+# working directory
+WORK="$HOME/tmp/nifi-release-$NIFI_VERSION"
+
+# handle tilde expansion
+WORK="${WORK/#\~/$HOME}"
+
+# warn the user if the working directory exists
+if [ -d "$WORK" ]; then
+ print_error "Directory $WORK exists, please rename it and start over"
+ exit 1
+fi
+
+if [ ! -d "$WORK" ]; then
+ mkdir -p "$WORK"
+fi
+print_item "Working directory is $WORK"
+
+print_section_header "Validation"
+
+KEYS="$NIFI_DEV_DIST/KEYS"
+LOCAL_ASSEMBLY="$NIFI_NAME_LOWER-$NIFI_VERSION-source-release.zip"
+NIFI_ASSEMBLY="$NIFI_RC_DIST$NIFI_NAME_LOWER-$NIFI_VERSION-source-release.zip"
+NIFI_ASSEMBLY_SIG="$NIFI_ASSEMBLY.asc"
+NIFI_ASSEMBLY_SHA1="$NIFI_ASSEMBLY.sha1"
+NIFI_ASSEMBLY_SHA256="$NIFI_ASSEMBLY.sha256"
+NIFI_ASSEMBLY_SHA512="$NIFI_ASSEMBLY.sha512"
+
+print_section_item "Downloading $KEYS"
+if ! wget -P "$WORK" "$KEYS" ; then
+ print_error "Failed to download $KEYS"
+ exit 1
+fi
+
+print_section_item "Downloading $NIFI_ASSEMBLY"
+if ! wget -P "$WORK" "$NIFI_ASSEMBLY" ; then
+ print_error "Failed to download $NIFI_ASSEMBLY"
+ exit 1
+fi
+print_section_item "Downloading $NIFI_ASSEMBLY_SIG"
+if ! wget -P "$WORK" "$NIFI_ASSEMBLY_SIG" ; then
+ print_error "Failed to download $NIFI_ASSEMBLY_SIG"
+ exit 1
+fi
+
+cd "$WORK" || exit 1
+print_section_item "importing nifi keys"
+
+if ! gpg --import KEYS ; then
+ print_error "failed to import KEYS"
+ exit 1
+fi
+
+print_section_item "Verifying Nifi Assembly"
+if ! gpg --verify ./"nifi-$NIFI_VERSION-source-release.zip.asc" "nifi-$NIFI_VERSION-source-release.zip" ; then
+ print_error "failed to verify Nifi Assembly"
+ exit 1
+fi
+
+LOCAL_SHA1="$(sha1sum ${LOCAL_ASSEMBLY} | cut -d' ' -f1)";
--- End diff --
I do not believe `sha1sum` is available on Mac OS X; use `shasum -a 1` instead.
---
[GitHub] nifi pull request #2806: NIFI-5068 Script to automate parts of RC validation
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2806#discussion_r198000501
--- Diff: dev-utilities/release-utilities/nifi-rc-check ---
@@ -0,0 +1,326 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+shopt -s nocasematch
+
+function print_help {
+ echo " "
+ echo "usage: ${0}"
+ echo " -v/--version=<version> The version of the Nifi release. [Required]"
--- End diff --
s/Nifi/NiFi/
---
[GitHub] nifi pull request #2806: NIFI-5068 Script to automate parts of RC validation
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2806#discussion_r198001332
--- Diff: dev-utilities/release-utilities/nifi-rc-check ---
@@ -0,0 +1,326 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+shopt -s nocasematch
+
+function print_help {
+ echo " "
+ echo "usage: ${0}"
+ echo " -v/--version=<version> The version of the Nifi release. [Required]"
+ echo " -c/--candidate=<RC#> Defines the Release Candidate. [Required]"
+ echo " -h/--help Usage information."
+ echo " "
+ echo "example: "
+ echo " nifi-rc-check --version=1.6.0 --candidate=RC2"
+ echo " "
+}
+
+print_section_header() {
+ echo
+ echo ----------------------------------------------
+ echo
+ echo "$@"
+ echo
+ echo ----------------------------------------------
+ echo
+}
+
+print_section_item() {
+ echo
+ echo "-----> $@"
+ echo
+}
+
+print_item() {
+ echo
+ echo "$@"
+ echo
+}
+
+print_error() {
+ echo
+ echo "ERROR !!!!!!"
+ echo ----------------------------------------------
+ echo
+ echo "$@"
+ echo
+ echo ----------------------------------------------
+ echo
+}
+
+NIFI_DEV_DIST="https://dist.apache.org/repos/dist/dev/nifi/"
+NIFI_DIST="https://www.apache.org/dist/nifi/"
+NIFI_NAME_LOWER="nifi";
+
+# print help, if the user just runs this without any args
+if [ "$#" -eq 0 ]; then
+ print_help
+ exit 1
+fi
+
+# handle command line options
+for i in "$@"; do
+ case ${i} in
+ #
+ # VERSION: The release version of Nifi to validate.
+ #
+ #
+ -v=*|--version=*)
+ VERSION="${i#*=}"
+ shift # past argument=value
+ ;;
+
+ #
+ # RC: Defines the RC# to use
+ #
+ # -c=RC2
+ # --candidate=RC2
+ #
+ -c=*|--candidate=*)
+ CANDIDATE="${i#*=}"
+ shift # past argument=value
+ ;;
+
+ #
+ # -h/--help
+ #
+ -h|--help)
+ help
+ exit 0
+ shift # past argument with no value
+ ;;
+
+ #
+ # Unknown option
+ #
+ *)
+ UNKNOWN_OPTION="${i#*=}"
+ echo "Error: unknown option: $UNKNOWN_OPTION"
+ help
+ ;;
+ esac
+done
+
+# validation
+if [ -z "$VERSION" ]; then
+ echo "Missing -v/--version is is required"
+ exit 1
+fi
+if [[ "$VERSION" =~ ^[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2} ]]; then
+ NIFI_VERSION="$VERSION"
+else
+ print_error "$VERSION may not be a valid version number"
+ exit 1
+fi
+
+if [ -z "$CANDIDATE" ]; then
+ echo "Missing -c/--candidate which is required"
+ exit 1
+fi
+
+if [[ "$CANDIDATE" =~ ^RC[0-9]+ ]]; then
+ RC=$(echo "$CANDIDATE" | tr '[:upper:]' '[:lower:]')
+ UPPER_RC=$(echo "$CANDIDATE" | tr '[:lower:]' '[:upper:]')
+elif [[ "$CANDIDATE" =~ ^[0-9]+ ]]; then
+ RC=rc"$CANDIDATE"
+ UPPER_RC=RC"$CANDIDATE"
+else
+ print_error "invalid RC, valid is RC# or just #"
+ exit 1
+fi
+
+print_section_header "Apache Nifi Version $NIFI_VERSION $UPPER_RC"
+
+NIFI_REL_RC="$NIFI_NAME_LOWER-$NIFI_VERSION-$UPPER_RC"
+
+NIFI_RC_DIST="$NIFI_DEV_DIST$NIFI_NAME_LOWER-$NIFI_VERSION/"
+print_item "Apache Nifi RC Distribution Root is $NIFI_RC_DIST"
+
+
+# working directory
+WORK="$HOME/tmp/nifi-release-$NIFI_VERSION"
+
+# handle tilde expansion
+WORK="${WORK/#\~/$HOME}"
+
+# warn the user if the working directory exists
+if [ -d "$WORK" ]; then
+ print_error "Directory $WORK exists, please rename it and start over"
+ exit 1
+fi
+
+if [ ! -d "$WORK" ]; then
+ mkdir -p "$WORK"
+fi
+print_item "Working directory is $WORK"
+
+print_section_header "Validation"
+
+KEYS="$NIFI_DEV_DIST/KEYS"
+LOCAL_ASSEMBLY="$NIFI_NAME_LOWER-$NIFI_VERSION-source-release.zip"
+NIFI_ASSEMBLY="$NIFI_RC_DIST$NIFI_NAME_LOWER-$NIFI_VERSION-source-release.zip"
+NIFI_ASSEMBLY_SIG="$NIFI_ASSEMBLY.asc"
+NIFI_ASSEMBLY_SHA1="$NIFI_ASSEMBLY.sha1"
+NIFI_ASSEMBLY_SHA256="$NIFI_ASSEMBLY.sha256"
+NIFI_ASSEMBLY_SHA512="$NIFI_ASSEMBLY.sha512"
+
+print_section_item "Downloading $KEYS"
+if ! wget -P "$WORK" "$KEYS" ; then
+ print_error "Failed to download $KEYS"
+ exit 1
+fi
+
+print_section_item "Downloading $NIFI_ASSEMBLY"
+if ! wget -P "$WORK" "$NIFI_ASSEMBLY" ; then
+ print_error "Failed to download $NIFI_ASSEMBLY"
+ exit 1
+fi
+print_section_item "Downloading $NIFI_ASSEMBLY_SIG"
+if ! wget -P "$WORK" "$NIFI_ASSEMBLY_SIG" ; then
+ print_error "Failed to download $NIFI_ASSEMBLY_SIG"
+ exit 1
+fi
+
+cd "$WORK" || exit 1
+print_section_item "importing nifi keys"
+
+if ! gpg --import KEYS ; then
+ print_error "failed to import KEYS"
+ exit 1
+fi
+
+print_section_item "Verifying Nifi Assembly"
+if ! gpg --verify ./"nifi-$NIFI_VERSION-source-release.zip.asc" "nifi-$NIFI_VERSION-source-release.zip" ; then
+ print_error "failed to verify Nifi Assembly"
+ exit 1
+fi
+
+LOCAL_SHA1="$(sha1sum ${LOCAL_ASSEMBLY} | cut -d' ' -f1)";
+REMOTE_SHA1="$(curl -s ${NIFI_ASSEMBLY_SHA1})"
+print_section_item "Verifying Nifi Assembly SHA1"
+if [ "${LOCAL_SHA1}" = "${REMOTE_SHA1}" ] ; then
+ echo "SHA-1 match"
+else
+ print_error "SHA-1 does not match : sha1sum: ${LOCAL_SHA1} Remote sha1sum: ${REMOTE_SHA1}"
+ exit 1
+fi
+
+LOCAL_SHA256=$(shasum -a256 ${LOCAL_ASSEMBLY} | cut -d' ' -f1);
+REMOTE_SHA256=$(curl -s ${NIFI_ASSEMBLY_SHA256})
+print_section_item "Verifying Nifi Assembly SHA256"
+if [ "${LOCAL_SHA256}" = "${REMOTE_SHA256}" ] ; then
+ echo "SHA-256 match"
+else
+ echo
+ echo
+ echo sha256sum: $(shasum -a256 ${LOCAL_ASSEMBLY} | cut -d' ' -f1);
+ echo Remote sha256sum: $(curl -s ${NIFI_ASSEMBLY_SHA256})
+ print_error "SHA-256 does not match"
+ exit 1
+fi
+
+LOCAL_SHA512=$(shasum -a512 ${LOCAL_ASSEMBLY} | cut -d' ' -f1);
+REMOTE_SHA512=$(curl -s ${NIFI_ASSEMBLY_SHA512})
+print_section_item "Verifying Nifi Assembly SHA512"
+if [ "${LOCAL_SHA512}" = "${REMOTE_SHA512}" ] ; then
+ echo "SHA-512 match"
+else
+ echo sha256sum: $(shasum -a512 ${LOCAL_ASSEMBLY} | cut -d' ' -f1);
+ echo Remote sha256sum: $(curl -s ${NIFI_ASSEMBLY_SHA512})
+ print_error "SHA-512 does not match"
+ exit 1
+fi
+
+print_section_item "Unpacking Assemblies"
+if ! unzip -q ${LOCAL_ASSEMBLY}; then
+ print_error "failed to unpack Nifi Assembly"
+ exit 1
+fi
+
+print_section_item "Retrieving remote Nifi repository"
+if ! git clone https://git-wip-us.apache.org/repos/asf/nifi.git; then
+ print_error "failed git clone https://git-wip-us.apache.org/repos/asf/nifi.git"
+ exit 1
+fi
+
+print_section_item "Getting the commit hash for ${UPPER_RC}"
+#git ls-remote --tags --refs https://github.com/apache/nifi.git | grep nifi-1.7.0-RC1 | cut -f1
+COMMIT_HASH=$(git ls-remote --tags --refs https://github.com/apache/nifi.git | grep "${NIFI_REL_RC}" | cut -f1);
+
+print_section_item "Commit Hash for ${NIFI_REL_RC} is ${COMMIT_HASH}"
+
+print_section_item "Checking out the correct tag"
+if ! git -C nifi checkout "${COMMIT_HASH}" ; then
+ print_error "failed git -C nifi checkout ${COMMIT_HASH}"
+ exit 1
+fi
+
+print_section_item "Checking for differences between downloaded rc sources and git"
+
+DIFF=$(diff --brief -r nifi-${NIFI_VERSION} nifi | grep -v DEPENDENCIES | grep -v .git* | grep -v project.basedir);
+
+if [ ! -z "${DIFF}" ] ; then
+ echo "${DIFF}"
+ print_error "[ERROR] There are differences!"
+ exit 1
+fi
+
+print_section_header "Build and Test"
+echo ""
+echo ""
+read -p " Build Nifi and run contrib check? [yN]" -n 1 -r
+echo
+if [[ $REPLY =~ ^[Yy]$ ]]; then
+ print_section_item "Performing build with contrib-check"
+ if ! mvn -f nifi-${NIFI_VERSION}/pom.xml clean install -Pcontrib-check ; then
--- End diff --
I think this should include `-Pinclude-grpc` for verifying the build during a release vote.
---
[GitHub] nifi pull request #2806: NIFI-5068 Script to automate parts of RC validation
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2806#discussion_r198000745
--- Diff: dev-utilities/release-utilities/nifi-rc-check ---
@@ -0,0 +1,326 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+shopt -s nocasematch
+
+function print_help {
+ echo " "
+ echo "usage: ${0}"
+ echo " -v/--version=<version> The version of the Nifi release. [Required]"
+ echo " -c/--candidate=<RC#> Defines the Release Candidate. [Required]"
+ echo " -h/--help Usage information."
+ echo " "
+ echo "example: "
+ echo " nifi-rc-check --version=1.6.0 --candidate=RC2"
+ echo " "
+}
+
+print_section_header() {
+ echo
+ echo ----------------------------------------------
+ echo
+ echo "$@"
+ echo
+ echo ----------------------------------------------
+ echo
+}
+
+print_section_item() {
+ echo
+ echo "-----> $@"
+ echo
+}
+
+print_item() {
+ echo
+ echo "$@"
+ echo
+}
+
+print_error() {
+ echo
+ echo "ERROR !!!!!!"
+ echo ----------------------------------------------
+ echo
+ echo "$@"
+ echo
+ echo ----------------------------------------------
+ echo
+}
+
+NIFI_DEV_DIST="https://dist.apache.org/repos/dist/dev/nifi/"
+NIFI_DIST="https://www.apache.org/dist/nifi/"
+NIFI_NAME_LOWER="nifi";
+
+# print help, if the user just runs this without any args
+if [ "$#" -eq 0 ]; then
+ print_help
+ exit 1
+fi
+
+# handle command line options
+for i in "$@"; do
+ case ${i} in
+ #
+ # VERSION: The release version of Nifi to validate.
+ #
+ #
+ -v=*|--version=*)
+ VERSION="${i#*=}"
+ shift # past argument=value
+ ;;
+
+ #
+ # RC: Defines the RC# to use
+ #
+ # -c=RC2
+ # --candidate=RC2
+ #
+ -c=*|--candidate=*)
+ CANDIDATE="${i#*=}"
+ shift # past argument=value
+ ;;
+
+ #
+ # -h/--help
+ #
+ -h|--help)
+ help
+ exit 0
+ shift # past argument with no value
+ ;;
+
+ #
+ # Unknown option
+ #
+ *)
+ UNKNOWN_OPTION="${i#*=}"
+ echo "Error: unknown option: $UNKNOWN_OPTION"
+ help
+ ;;
+ esac
+done
+
+# validation
+if [ -z "$VERSION" ]; then
+ echo "Missing -v/--version is is required"
+ exit 1
+fi
+if [[ "$VERSION" =~ ^[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2} ]]; then
+ NIFI_VERSION="$VERSION"
+else
+ print_error "$VERSION may not be a valid version number"
+ exit 1
+fi
+
+if [ -z "$CANDIDATE" ]; then
+ echo "Missing -c/--candidate which is required"
+ exit 1
+fi
+
+if [[ "$CANDIDATE" =~ ^RC[0-9]+ ]]; then
+ RC=$(echo "$CANDIDATE" | tr '[:upper:]' '[:lower:]')
+ UPPER_RC=$(echo "$CANDIDATE" | tr '[:lower:]' '[:upper:]')
+elif [[ "$CANDIDATE" =~ ^[0-9]+ ]]; then
+ RC=rc"$CANDIDATE"
+ UPPER_RC=RC"$CANDIDATE"
+else
+ print_error "invalid RC, valid is RC# or just #"
+ exit 1
+fi
+
+print_section_header "Apache Nifi Version $NIFI_VERSION $UPPER_RC"
+
+NIFI_REL_RC="$NIFI_NAME_LOWER-$NIFI_VERSION-$UPPER_RC"
+
+NIFI_RC_DIST="$NIFI_DEV_DIST$NIFI_NAME_LOWER-$NIFI_VERSION/"
+print_item "Apache Nifi RC Distribution Root is $NIFI_RC_DIST"
+
+
+# working directory
+WORK="$HOME/tmp/nifi-release-$NIFI_VERSION"
+
+# handle tilde expansion
+WORK="${WORK/#\~/$HOME}"
+
+# warn the user if the working directory exists
+if [ -d "$WORK" ]; then
+ print_error "Directory $WORK exists, please rename it and start over"
+ exit 1
+fi
+
+if [ ! -d "$WORK" ]; then
+ mkdir -p "$WORK"
+fi
+print_item "Working directory is $WORK"
+
+print_section_header "Validation"
+
+KEYS="$NIFI_DEV_DIST/KEYS"
+LOCAL_ASSEMBLY="$NIFI_NAME_LOWER-$NIFI_VERSION-source-release.zip"
+NIFI_ASSEMBLY="$NIFI_RC_DIST$NIFI_NAME_LOWER-$NIFI_VERSION-source-release.zip"
+NIFI_ASSEMBLY_SIG="$NIFI_ASSEMBLY.asc"
+NIFI_ASSEMBLY_SHA1="$NIFI_ASSEMBLY.sha1"
+NIFI_ASSEMBLY_SHA256="$NIFI_ASSEMBLY.sha256"
+NIFI_ASSEMBLY_SHA512="$NIFI_ASSEMBLY.sha512"
+
+print_section_item "Downloading $KEYS"
+if ! wget -P "$WORK" "$KEYS" ; then
+ print_error "Failed to download $KEYS"
+ exit 1
+fi
+
+print_section_item "Downloading $NIFI_ASSEMBLY"
+if ! wget -P "$WORK" "$NIFI_ASSEMBLY" ; then
+ print_error "Failed to download $NIFI_ASSEMBLY"
+ exit 1
+fi
+print_section_item "Downloading $NIFI_ASSEMBLY_SIG"
+if ! wget -P "$WORK" "$NIFI_ASSEMBLY_SIG" ; then
+ print_error "Failed to download $NIFI_ASSEMBLY_SIG"
+ exit 1
+fi
+
+cd "$WORK" || exit 1
+print_section_item "importing nifi keys"
+
+if ! gpg --import KEYS ; then
+ print_error "failed to import KEYS"
+ exit 1
+fi
+
+print_section_item "Verifying Nifi Assembly"
+if ! gpg --verify ./"nifi-$NIFI_VERSION-source-release.zip.asc" "nifi-$NIFI_VERSION-source-release.zip" ; then
+ print_error "failed to verify Nifi Assembly"
--- End diff --
This should use `gpg --verify -v ...` to ensure that the internal digest algorithm being used for the signature is SHA-512.
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by joewitt <gi...@git.apache.org>.
Github user joewitt commented on the issue:
https://github.com/apache/nifi/pull/2806
@ottobackwards i'd recommend just moving this to be a bash script made available on the wiki and close the PR.
You could
1. Propose an edit to the release process to make this available as an alternative way for folks who want to validate the RC (as an alternative to the current path)
2. Put this bash script on the wiki and link to it from #1.
Thanks
---
[GitHub] nifi pull request #2806: NIFI-5068 Script to automate parts of RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards closed the pull request at:
https://github.com/apache/nifi/pull/2806
---
[GitHub] nifi issue #2806: NIFI-5068 Script to automate parts of RC validation
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/nifi/pull/2806
@alopresto I have made the suggested changes. Unfortunately the build has been pulled, so the script won't run until next release.
---