You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "RussellSpitzer (via GitHub)" <gi...@apache.org> on 2026/05/06 21:09:01 UTC

[PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

RussellSpitzer opened a new pull request, #3548:
URL: https://github.com/apache/parquet-java/pull/3548

   
   
   Includes 85 bats unit tests covering all shared libraries.
   
   <!--
   Thanks for opening a pull request!
   
   If you're new to Parquet-Java, information on how to contribute can be found here: https://parquet.apache.org/docs/contribution-guidelines/contributing
   
   Please open a GitHub issue for this pull request: https://github.com/apache/parquet-java/issues/new/choose
   and format pull request title as below:
   
       GH-${GITHUB_ISSUE_ID}: ${SUMMARY}
   
   or simply use the title below if it is a minor issue:
   
       MINOR: ${SUMMARY}
   
   -->
   
   ### Rationale for this change
   Adds a release automation framework modeled after Apache Polaris, adapted for Parquet's Maven-based build. Replaces the manual maven-release-plugin workflow with explicit, scriptable steps that support both CI (GitHub Actions) and local execution, with dry-run by default.
   
   ### What changes are included in this PR?
   Scripts (release/bin/):
   - prepare-rc.sh: full pre-vote flow (branch, version, tag, Nexus, SVN, GitHub pre-release, vote email)
   - publish-release.sh: full post-vote flow (SVN promotion, final tag, Nexus release, GitHub release, version bump, announce email)
   - cancel-rc.sh: rollback a failed RC (Nexus drop, SVN cleanup)
   
   Shared libraries (release/libs/):
   - _constants.sh, _log.sh, _exec.sh, _version.sh
   - _github.sh, _nexus.sh, _maven.sh
   
   GitHub Actions workflows:
   - release-prepare-rc.yml, release-publish.yml, release-cancel-rc.yml
   - ci-release-scripts.yml (bats unit tests on PR/push)
   
   ### Are these changes tested?
   Only locally and with fake commands. **To actually make this work we also have to raise an Infra ticket to get appropriate secrets applied to the parquet-java repo.**
   
   ### Are there any user-facing changes?
   No, just for contributors
   
   
   ----
   
   ## Release Workflow
   
   This PR replaces the manual release process (documented in
   [How to Release](https://parquet.apache.org/docs/contribution-guidelines/releasing/)
   and the existing `dev/` scripts: `prepare-release.sh`, `source-release.sh`,
   `finalize-release`) with three GitHub Actions workflows backed by locally-runnable
   Bash scripts. All workflows default to **dry-run mode**.
   
   ### 1. Prepare RC (Pre-Vote)
   
   The release manager launches the **"Prepare Release Candidate"** workflow
   (`release-prepare-rc.yml`) via `workflow_dispatch` with:
   
   - **version** — release version (e.g. `1.18.0`)
   - **rc_number** *(optional)* — override RC number (auto-detected if empty)
   - **dry_run** — defaults to `true`; set to `false` for real execution
   
   | # | Step in `prepare-rc.sh` | Replaces (docs / existing script) |
   |---|-------------------------|-----------------------------------|
   | 0 | Validate inputs | *New* — checks version format, verifies `gpg`/`svn`/`mvnw` |
   | 1 | Create release branch | Manual `git branch parquet-X.Y.x` |
   | 2 | Auto-detect RC number | Manual decision by release manager |
   | 3 | Verify CI checks | Manual check of GitHub Actions UI |
   | 4 | Set POM versions | `dev/prepare-release.sh` (`mvn release:prepare`) |
   | 5 | Create RC tag and push | `dev/prepare-release.sh` (`mvn release:prepare`) |
   | 6 | Deploy to Nexus | Manual `mvn release:perform` (prompted by `prepare-release.sh`) |
   | 7 | Build source tarball | `dev/source-release.sh` (`git archive`, `gpg`, `shasum`) |
   | 8 | Stage to SVN dist/dev | `dev/source-release.sh` (`svn co`, `svn add`, `svn ci`) |
   | 9 | Create GitHub pre-release | *New* — creates a GitHub pre-release with auto-generated notes |
   
   The script also generates a `[VOTE]` email template with all links and
   hashes. The release manager copies this and **manually sends it** to
   `dev@parquet.apache.org`.
   
   ### 2. Vote
   
   The release manager waits 72 hours for the community vote and tallies the
   results manually.
   
   ---
   
   ### If the vote fails → Cancel RC
   
   The release manager launches the **"Cancel Release Candidate"** workflow
   (`release-cancel-rc.yml`) via `workflow_dispatch` with:
   
   - **version** — release version (e.g. `1.18.0`)
   - **rc_number** — the RC number to cancel (e.g. `1`)
   - **staging_repo_id** — Nexus staging repository ID (e.g. `orgapacheparquet-1234`)
   - **dry_run** — defaults to `true`
   
   | # | Step in `cancel-rc.sh` | Replaces (docs / existing script) |
   |---|------------------------|-----------------------------------|
   | 1 | Drop Nexus staging repo | Manual Nexus UI operation |
   | 2 | Delete SVN artifacts from dist/dev | Manual `svn rm` |
   
   The script also generates a `[RESULT][VOTE]` failure email template.
   The release manager fills in the failure reason and **manually sends it**.
   
   ---
   
   ### If the vote passes → Publish Release
   
   The release manager launches the **"Publish Release"** workflow
   (`release-publish.yml`) via `workflow_dispatch` with:
   
   - **version** — release version (e.g. `1.18.0`)
   - **rc_number** *(optional)* — RC that passed the vote (auto-detects latest if empty; rejects older RCs)
   - **staging_repo_id** — Nexus staging repository ID (e.g. `orgapacheparquet-1234`)
   - **next_dev_version** — next development version without `-SNAPSHOT` (e.g. `1.18.1`)
   - **dry_run** — defaults to `true`
   
   | # | Step in `publish-release.sh` | Replaces (docs / existing script) |
   |---|------------------------------|-----------------------------------|
   | 1 | SVN Promotion | Manual `svn mv` from `dist/dev` to `dist/release` |
   | 2 | Old Release Cleanup | Manual `svn rm` of prior versions |
   | 3 | Create final release tag | `dev/finalize-release` (`git tag`) |
   | 4 | Release Nexus staging repo | Manual Nexus UI "Release" button |
   | 5 | Create GitHub Release | Manual GitHub UI release creation |
   | 6 | Bump to next dev version | `dev/finalize-release` (`mvn release:update-versions`, `versions:set-property`, `git commit`) |
   
   ### 3. Manual Follow-ups
   
   After the publish workflow completes, the release manager must:
   
   1. **Send the `[ANNOUNCE]` email** (generated by step 7 of `publish-release.sh`) to `dev@parquet.apache.org` and `announce@apache.org`
   2. **Create a PR against [`apache/parquet-site`](https://github.com/apache/parquet-site)** on the `staging` branch with the blog post template (generated by step 8 of `publish-release.sh`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3288423328


##########
release/bin/publish-release.sh:
##########
@@ -0,0 +1,408 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+LIBS_DIR="${SCRIPT_DIR}/../libs"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_log.sh"
+source "${LIBS_DIR}/_exec.sh"
+source "${LIBS_DIR}/_version.sh"
+source "${LIBS_DIR}/_nexus.sh"
+source "${LIBS_DIR}/_svn.sh"
+
+# ---------------------------------------------------------------------------
+# Usage
+# ---------------------------------------------------------------------------
+function usage {
+  cat <<EOF
+Usage: $0 <version> <staging-repo-id> [--rc <num>] [--allow-description-mismatch]
+
+Publish a release after the vote passes.
+
+Arguments:
+  version               Release version (e.g., 1.18.0)
+  staging-repo-id       Nexus staging repository ID (e.g., orgapacheparquet-1234)
+
+Options:
+  --rc <num>            RC number that passed the vote (default: auto-detect latest)
+  --allow-description-mismatch
+                        Bypass the staging-repo description check (for recovery scenarios)
+  --help                Show this help
+
+Before any destructive action, this script verifies that the staging repo
+belongs to org.apache.parquet, is in 'closed' state, contains
+${NEXUS_VERIFY_ARTIFACT_ID:-parquet-common}-<version>.pom, and has a
+description matching "Apache Parquet <version> RC<num>".
+
+The next development version is auto-computed by incrementing the patch
+version (e.g., 1.18.0 -> 1.18.1-SNAPSHOT).
+
+Environment variables:
+  DRY_RUN               Set to 0 for real execution (default: 1)
+  NEXUS_USERNAME        Apache Nexus username
+  NEXUS_PASSWORD        Apache Nexus password
+  SVN_USERNAME          SVN username for dist.apache.org
+  SVN_PASSWORD          SVN password
+  GITHUB_TOKEN          GitHub token for release creation
+
+Example:
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234 --rc 2
+EOF
+  exit "${1:-0}"
+}
+
+# ---------------------------------------------------------------------------
+# Parse arguments
+# ---------------------------------------------------------------------------
+version=""
+staging_repo_id=""
+rc_num=""
+allow_description_mismatch=0
+positional=()
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --rc)
+      if [[ -z "${2:-}" ]]; then
+        print_error "--rc requires a value"
+        usage 1
+      fi
+      rc_num="$2"
+      shift 2
+      ;;
+    --allow-description-mismatch)
+      allow_description_mismatch=1
+      shift
+      ;;
+    --help|-h)
+      usage 0
+      ;;
+    -*)
+      print_error "Unknown option: $1"
+      usage 1
+      ;;
+    *)
+      positional+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [[ ${#positional[@]} -lt 2 ]]; then
+  print_error "Expected 2 positional arguments (version, staging-repo-id), got ${#positional[@]}"
+  usage 1
+fi
+
+version="${positional[0]}"
+staging_repo_id="${positional[1]}"
+
+# ---------------------------------------------------------------------------
+# Validate inputs
+# ---------------------------------------------------------------------------
+step_summary "## Release Publication"
+step_summary ""
+
+if [[ ${DRY_RUN:-1} -eq 1 ]]; then
+  step_summary "> **DRY RUN** -- no changes will be made"
+  step_summary ""
+fi
+
+if ! validate_and_extract_version "${version}"; then
+  print_error "Invalid version format: '${version}'"
+  exit 1
+fi
+
+next_dev_version="${major}.${minor}.$(( patch + 1 ))"
+
+if ! [[ "${staging_repo_id}" =~ ^[a-zA-Z][a-zA-Z0-9._-]*$ ]]; then
+  print_error "Invalid staging repository ID: '${staging_repo_id}'. Expected alphanumeric with dots/hyphens (e.g., orgapacheparquet-1234)."
+  exit 1
+fi
+
+if [[ -z "${rc_num}" ]]; then
+  print_info "No RC number specified, auto-detecting latest RC for ${version}..."
+  if ! find_latest_rc_number "${version}"; then
+    exit 1
+  fi
+  rc_num="${latest_rc_number}"
+  print_info "Auto-detected latest RC: rc${rc_num}"
+else
+  if ! [[ "${rc_num}" =~ ^[0-9]+$ ]]; then
+    print_error "Invalid RC number: '${rc_num}'. Expected a non-negative integer."
+    exit 1
+  fi
+
+  if find_latest_rc_number "${version}" 2>/dev/null; then
+    if [[ "${rc_num}" -ne "${latest_rc_number}" ]]; then
+      print_error "RC${rc_num} is not the latest RC for ${version}. Latest is rc${latest_rc_number}."
+      print_error "Publishing an older RC is likely a mistake. If intentional, delete the newer RC tags first."
+      exit 1
+    fi
+  fi
+fi
+
+rc_tag="${TAG_PREFIX}${version}-rc${rc_num}"
+final_tag="${TAG_PREFIX}${version}"
+
+if ! git rev-parse "${rc_tag}" >/dev/null 2>&1; then
+  print_error "RC tag ${rc_tag} does not exist"
+  exit 1
+fi
+
+rc_commit=$(git rev-list -1 "${rc_tag}")
+current_commit=$(git rev-parse HEAD)
+
+if [[ "${current_commit}" != "${rc_commit}" ]]; then
+  print_error "Current HEAD (${current_commit}) does not match RC tag ${rc_tag} (${rc_commit})"
+  print_error "The release branch has commits beyond the voted RC. Either reset the branch or create a new RC."
+  exit 1
+fi
+
+if git rev-parse "${final_tag}" >/dev/null 2>&1; then
+  print_error "Final release tag ${final_tag} already exists"
+  exit 1
+fi
+
+step_summary "| Parameter | Value |"
+step_summary "| --- | --- |"
+step_summary "| Version | \`${version}\` |"
+step_summary "| RC tag | \`${rc_tag}\` |"
+step_summary "| Final tag | \`${final_tag}\` |"
+step_summary "| Staging repo | \`${staging_repo_id}\` |"
+step_summary "| Next dev version | \`${next_dev_version}-SNAPSHOT\` |"
+step_summary "| Commit | \`${rc_commit}\` |"
+
+# ---------------------------------------------------------------------------
+# Step 0: Verify staging repository before any destructive action
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Staging Repository Verification"
+
+if ! nexus_verify_staging_repo "${staging_repo_id}" "${version}" "${rc_num}" "${allow_description_mismatch}"; then
+  step_summary "Staging repository verification: **FAILED**"
+  exit 1
+fi
+step_summary "Staging repository \`${staging_repo_id}\` verified"
+
+# ---------------------------------------------------------------------------
+# Step 1: Move SVN artifacts from dist/dev to dist/release
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### SVN Promotion"
+
+svn_promote_rc_to_release "${version}" "${rc_num}"
+
+step_summary "Promoted \`${rc_tag}\` -> \`${final_tag}\` on dist.apache.org"
+
+# ---------------------------------------------------------------------------
+# Step 2: Clean up superseded patch releases on the same minor branch
+# ---------------------------------------------------------------------------
+# Policy: only patch releases of the *same* minor (parquet-${major}.${minor}.x)

Review Comment:
   Thanks Russell, that makes sense. I think it would be good to clean up the old versions 👍 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-5258590033

   Thanks! I'll ping Infra to see if we can get those secrets set up.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3225424719


##########
release/bin/publish-release.sh:
##########
@@ -0,0 +1,408 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+LIBS_DIR="${SCRIPT_DIR}/../libs"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_log.sh"
+source "${LIBS_DIR}/_exec.sh"
+source "${LIBS_DIR}/_version.sh"
+source "${LIBS_DIR}/_nexus.sh"
+source "${LIBS_DIR}/_svn.sh"
+
+# ---------------------------------------------------------------------------
+# Usage
+# ---------------------------------------------------------------------------
+function usage {
+  cat <<EOF
+Usage: $0 <version> <staging-repo-id> [--rc <num>] [--allow-description-mismatch]
+
+Publish a release after the vote passes.
+
+Arguments:
+  version               Release version (e.g., 1.18.0)
+  staging-repo-id       Nexus staging repository ID (e.g., orgapacheparquet-1234)
+
+Options:
+  --rc <num>            RC number that passed the vote (default: auto-detect latest)
+  --allow-description-mismatch
+                        Bypass the staging-repo description check (for recovery scenarios)
+  --help                Show this help
+
+Before any destructive action, this script verifies that the staging repo
+belongs to org.apache.parquet, is in 'closed' state, contains
+${NEXUS_VERIFY_ARTIFACT_ID:-parquet-common}-<version>.pom, and has a
+description matching "Apache Parquet <version> RC<num>".
+
+The next development version is auto-computed by incrementing the patch
+version (e.g., 1.18.0 -> 1.18.1-SNAPSHOT).
+
+Environment variables:
+  DRY_RUN               Set to 0 for real execution (default: 1)
+  NEXUS_USERNAME        Apache Nexus username
+  NEXUS_PASSWORD        Apache Nexus password
+  SVN_USERNAME          SVN username for dist.apache.org
+  SVN_PASSWORD          SVN password
+  GITHUB_TOKEN          GitHub token for release creation
+
+Example:
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234 --rc 2
+EOF
+  exit "${1:-0}"
+}
+
+# ---------------------------------------------------------------------------
+# Parse arguments
+# ---------------------------------------------------------------------------
+version=""
+staging_repo_id=""
+rc_num=""
+allow_description_mismatch=0
+positional=()
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --rc)
+      if [[ -z "${2:-}" ]]; then
+        print_error "--rc requires a value"
+        usage 1
+      fi
+      rc_num="$2"
+      shift 2
+      ;;
+    --allow-description-mismatch)
+      allow_description_mismatch=1
+      shift
+      ;;
+    --help|-h)
+      usage 0
+      ;;
+    -*)
+      print_error "Unknown option: $1"
+      usage 1
+      ;;
+    *)
+      positional+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [[ ${#positional[@]} -lt 2 ]]; then
+  print_error "Expected 2 positional arguments (version, staging-repo-id), got ${#positional[@]}"
+  usage 1
+fi
+
+version="${positional[0]}"
+staging_repo_id="${positional[1]}"
+
+# ---------------------------------------------------------------------------
+# Validate inputs
+# ---------------------------------------------------------------------------
+step_summary "## Release Publication"
+step_summary ""
+
+if [[ ${DRY_RUN:-1} -eq 1 ]]; then
+  step_summary "> **DRY RUN** -- no changes will be made"
+  step_summary ""
+fi
+
+if ! validate_and_extract_version "${version}"; then
+  print_error "Invalid version format: '${version}'"
+  exit 1
+fi
+
+next_dev_version="${major}.${minor}.$(( patch + 1 ))"
+
+if ! [[ "${staging_repo_id}" =~ ^[a-zA-Z][a-zA-Z0-9._-]*$ ]]; then
+  print_error "Invalid staging repository ID: '${staging_repo_id}'. Expected alphanumeric with dots/hyphens (e.g., orgapacheparquet-1234)."
+  exit 1
+fi
+
+if [[ -z "${rc_num}" ]]; then
+  print_info "No RC number specified, auto-detecting latest RC for ${version}..."
+  if ! find_latest_rc_number "${version}"; then
+    exit 1
+  fi
+  rc_num="${latest_rc_number}"
+  print_info "Auto-detected latest RC: rc${rc_num}"
+else
+  if ! [[ "${rc_num}" =~ ^[0-9]+$ ]]; then
+    print_error "Invalid RC number: '${rc_num}'. Expected a non-negative integer."
+    exit 1
+  fi
+
+  if find_latest_rc_number "${version}" 2>/dev/null; then
+    if [[ "${rc_num}" -ne "${latest_rc_number}" ]]; then
+      print_error "RC${rc_num} is not the latest RC for ${version}. Latest is rc${latest_rc_number}."
+      print_error "Publishing an older RC is likely a mistake. If intentional, delete the newer RC tags first."
+      exit 1
+    fi
+  fi
+fi
+
+rc_tag="${TAG_PREFIX}${version}-rc${rc_num}"
+final_tag="${TAG_PREFIX}${version}"
+
+if ! git rev-parse "${rc_tag}" >/dev/null 2>&1; then
+  print_error "RC tag ${rc_tag} does not exist"
+  exit 1
+fi
+
+rc_commit=$(git rev-list -1 "${rc_tag}")
+current_commit=$(git rev-parse HEAD)
+
+if [[ "${current_commit}" != "${rc_commit}" ]]; then
+  print_error "Current HEAD (${current_commit}) does not match RC tag ${rc_tag} (${rc_commit})"
+  print_error "The release branch has commits beyond the voted RC. Either reset the branch or create a new RC."
+  exit 1
+fi
+
+if git rev-parse "${final_tag}" >/dev/null 2>&1; then
+  print_error "Final release tag ${final_tag} already exists"
+  exit 1
+fi
+
+step_summary "| Parameter | Value |"
+step_summary "| --- | --- |"
+step_summary "| Version | \`${version}\` |"
+step_summary "| RC tag | \`${rc_tag}\` |"
+step_summary "| Final tag | \`${final_tag}\` |"
+step_summary "| Staging repo | \`${staging_repo_id}\` |"
+step_summary "| Next dev version | \`${next_dev_version}-SNAPSHOT\` |"
+step_summary "| Commit | \`${rc_commit}\` |"
+
+# ---------------------------------------------------------------------------
+# Step 0: Verify staging repository before any destructive action
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Staging Repository Verification"
+
+if ! nexus_verify_staging_repo "${staging_repo_id}" "${version}" "${rc_num}" "${allow_description_mismatch}"; then
+  step_summary "Staging repository verification: **FAILED**"
+  exit 1
+fi
+step_summary "Staging repository \`${staging_repo_id}\` verified"
+
+# ---------------------------------------------------------------------------
+# Step 1: Move SVN artifacts from dist/dev to dist/release
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### SVN Promotion"
+
+svn_promote_rc_to_release "${version}" "${rc_num}"
+
+step_summary "Promoted \`${rc_tag}\` -> \`${final_tag}\` on dist.apache.org"
+
+# ---------------------------------------------------------------------------
+# Step 2: Clean up superseded patch releases on the same minor branch
+# ---------------------------------------------------------------------------
+# Policy: only patch releases of the *same* minor (parquet-${major}.${minor}.x)

Review Comment:
   asf prefer old stuff -> archive as it's not mirrored and reduces the amount which is replicated. Keeps the mirror sites happier



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-4422484668

   I like this, and the redaction is good. which is why those tests matter too: they verify that local redaction works.
   
   Don't see any documentation though; it does need to get covered in the releasing docs.
   
   I think it's good to consider mandating here the concept of having specific project release keys: your parquet signing key != your iceberg signing key != your polaris one, though they'll be signed by your personal key, cross trust each other etc.
   
   that way, if a private key is leaked
   * the fact that someone was using your parquet key to sign an iceberg release would be obvious
   * revoking the key only declares your parquet releases as untrusted, doesn't affect other projects, or any of your code commits.
   
   I had to revoke a key in 2017 and git log was very unhappy about how the history was now full of revoked key commits. https://steveloughran.blogspot.com/2017/10/roca-breaks-my-commit-process.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "wgtmac (via GitHub) via issues" <is...@parquet.apache.org>.
wgtmac commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r4504911349


##########
.github/workflows/release-publish.yml:
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Publish After Vote
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number that passed the vote (leave empty to auto-detect latest)'
+        required: false
+        type: string
+        default: ''
+      staging_repo_id:
+        description: 'Nexus staging repository ID (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      allow_description_mismatch:
+        description: 'Bypass the staging-repo description check (recovery only)'
+        required: false
+        type: boolean
+        default: false
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+concurrency:
+  group: release-${{ inputs.version }}
+  cancel-in-progress: false
+
+jobs:
+  publish-release:
+    if: github.repository_owner == 'apache'
+    name: Publish Release
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
+        with:
+          distribution: temurin
+          java-version: '11'
+
+      - name: Install Subversion
+        run: sudo apt-get update -q && sudo apt-get install -q -y subversion
+
+      - name: Configure Git
+        run: |
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Publish Release
+        env:
+          DRY_RUN: ${{ inputs.dry_run && '1' || '0' }}
+          NEXUS_USERNAME: ${{ secrets.PARQUET_NEXUS_USER }}
+          NEXUS_PASSWORD: ${{ secrets.PARQUET_NEXUS_PASSWORD }}
+          SVN_USERNAME: ${{ secrets.PARQUET_SVN_DEV_USERNAME }}

Review Comment:
   Publishing 1.18.2 RC2 failed: https://github.com/apache/parquet-java/actions/runs/42442927330
   
   ```
   svn mv .../dev/parquet/apache-parquet-1.18.2-rc2 \
          .../release/parquet/apache-parquet-1.18.2
   svn: E175013: Access to '/repos/dist/!svn/rvr/89131/dev/parquet/apache-parquet-1.18.2-rc2' forbidden
   ```
   
   It seems that this credential only works for the dev SVN not the release one. I've raised this to ASF Infra: https://issues.apache.org/jira/browse/INFRA-28245
   
   cc @manuzhang @RussellSpitzer @Fokko 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197523999


##########
.github/workflows/release-cancel-rc.yml:
##########
@@ -0,0 +1,74 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Cancel RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number to cancel (e.g., 0)'
+        required: true
+        type: string
+      staging_repo_id:
+        description: 'Nexus staging repository ID to drop (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+jobs:
+  cancel-rc:
+    name: Cancel Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Install Subversion
+        run: sudo apt-get update && sudo apt-get install -y subversion
+
+      - name: Cancel Release Candidate
+        env:
+          DRY_RUN: ${{ inputs.dry_run && '1' || '0' }}

Review Comment:
   anything in "secrets" is automatically redacted by Github Actions so we don't have to worry about any of this stuff being exposed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3220320078


##########
.github/workflows/ci-release-scripts.yml:
##########
@@ -0,0 +1,59 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Test Release Scripts
+
+on:
+  pull_request:
+    paths:
+      - 'release/**'
+  push:
+    branches:
+      - master
+    paths:
+      - 'release/**'
+
+jobs:

Review Comment:
   They aren't going to get much maintenance (hopefully), but will be a good regression test against changes in os images.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "Humbedooh (via GitHub) via issues" <is...@parquet.apache.org>.
Humbedooh commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r4526624520


##########
.github/workflows/release-publish.yml:
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Publish After Vote
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number that passed the vote (leave empty to auto-detect latest)'
+        required: false
+        type: string
+        default: ''
+      staging_repo_id:
+        description: 'Nexus staging repository ID (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      allow_description_mismatch:
+        description: 'Bypass the staging-repo description check (recovery only)'
+        required: false
+        type: boolean
+        default: false
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+concurrency:
+  group: release-${{ inputs.version }}
+  cancel-in-progress: false
+
+jobs:
+  publish-release:
+    if: github.repository_owner == 'apache'
+    name: Publish Release
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
+        with:
+          distribution: temurin
+          java-version: '11'
+
+      - name: Install Subversion
+        run: sudo apt-get update -q && sudo apt-get install -q -y subversion
+
+      - name: Configure Git
+        run: |
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Publish Release
+        env:
+          DRY_RUN: ${{ inputs.dry_run && '1' || '0' }}
+          NEXUS_USERNAME: ${{ secrets.PARQUET_NEXUS_USER }}
+          NEXUS_PASSWORD: ${{ secrets.PARQUET_NEXUS_PASSWORD }}
+          SVN_USERNAME: ${{ secrets.PARQUET_SVN_DEV_USERNAME }}

Review Comment:
   You cannot release through GitHub Actions, that is not a legal person. You can only stage release candidates through these role accounts. Releases have to be done by either a legal person in the project, or through sanctioned services like ATR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3288368423


##########
.github/workflows/release-cancel-rc.yml:
##########
@@ -0,0 +1,86 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Cancel RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number to cancel (e.g., 0)'
+        required: true
+        type: string
+      staging_repo_id:
+        description: 'Nexus staging repository ID to drop (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      allow_description_mismatch:
+        description: 'Bypass the staging-repo description check (recovery only)'
+        required: false
+        type: boolean
+        default: false
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+concurrency:
+  group: release-${{ inputs.version }}
+  cancel-in-progress: false
+
+jobs:
+  cancel-rc:
+    if: github.repository_owner == 'apache'
+    name: Cancel Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Install Subversion
+        run: sudo apt-get update && sudo apt-get install -y subversion

Review Comment:
   apt-get is pretty verbose, how about only showing the errors?
   ```suggestion
           run: sudo apt-get update -q && sudo apt-get install -q -y subversion
   ```



##########
.github/workflows/ci-release-scripts.yml:
##########
@@ -0,0 +1,64 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Test Release Scripts
+
+on:
+  pull_request:
+    paths:
+      - 'release/**'
+  push:
+    branches:
+      - master
+    paths:
+      - 'release/**'
+
+jobs:
+  bats:
+    name: Release Script Unit Tests
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Install bats-core
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y bats

Review Comment:
   apt-get is pretty chatty, how about running this with only showing the errors:
   ```suggestion
           run: sudo apt-get update -q && sudo apt-get install -q -y bats
   ```



##########
.github/workflows/release-prepare-rc.yml:
##########
@@ -0,0 +1,94 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Prepare RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number override (leave empty for auto-detect)'
+        required: false
+        type: string
+        default: ''
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+concurrency:
+  group: release-${{ inputs.version }}
+  cancel-in-progress: false
+
+jobs:
+  prepare-rc:
+    if: github.repository_owner == 'apache'
+    name: Prepare Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: '11'
+
+      - name: Import GPG key and configure Git
+        env:
+          GPG_PRIVATE_KEY: ${{ secrets.PARQUET_GPG_PRIVATE_KEY }}
+        run: |
+          echo "${GPG_PRIVATE_KEY}" | gpg --batch --import
+          KEY_ID=$(gpg --list-keys --with-colons | grep '^fpr' | head -1 | cut -d: -f10)
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+          git config --global user.signingkey "${KEY_ID}"
+          git config --global commit.gpgsign true
+
+      - name: Install Subversion
+        run: sudo apt-get update && sudo apt-get install -y subversion

Review Comment:
   apt-get is pretty verbose, how about only showing the errors?
   ```suggestion
           run: sudo apt-get update -q && sudo apt-get install -q -y subversion
   ```



##########
.github/workflows/release-publish.yml:
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Publish After Vote
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number that passed the vote (leave empty to auto-detect latest)'
+        required: false
+        type: string
+        default: ''
+      staging_repo_id:
+        description: 'Nexus staging repository ID (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      allow_description_mismatch:
+        description: 'Bypass the staging-repo description check (recovery only)'
+        required: false
+        type: boolean
+        default: false
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+concurrency:
+  group: release-${{ inputs.version }}
+  cancel-in-progress: false
+
+jobs:
+  publish-release:
+    if: github.repository_owner == 'apache'
+    name: Publish Release
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: '11'
+
+      - name: Install Subversion
+        run: sudo apt-get update && sudo apt-get install -y subversion

Review Comment:
   apt-get is pretty verbose, how about only showing the errors?
   ```suggestion
           run: sudo apt-get update -q && sudo apt-get install -q -y subversion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197521417


##########
.github/workflows/release-cancel-rc.yml:
##########
@@ -0,0 +1,74 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Cancel RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number to cancel (e.g., 0)'
+        required: true
+        type: string
+      staging_repo_id:
+        description: 'Nexus staging repository ID to drop (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+jobs:
+  cancel-rc:
+    name: Cancel Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4

Review Comment:
   Parquet isn't using [zimor](https://github.com/zizmorcore/zizmor) yet, but in the future we should switch this and all other actions to hard coded sha's  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3276087248


##########
release/bin/publish-release.sh:
##########
@@ -0,0 +1,408 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+LIBS_DIR="${SCRIPT_DIR}/../libs"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_log.sh"
+source "${LIBS_DIR}/_exec.sh"
+source "${LIBS_DIR}/_version.sh"
+source "${LIBS_DIR}/_nexus.sh"
+source "${LIBS_DIR}/_svn.sh"
+
+# ---------------------------------------------------------------------------
+# Usage
+# ---------------------------------------------------------------------------
+function usage {
+  cat <<EOF
+Usage: $0 <version> <staging-repo-id> [--rc <num>] [--allow-description-mismatch]
+
+Publish a release after the vote passes.
+
+Arguments:
+  version               Release version (e.g., 1.18.0)
+  staging-repo-id       Nexus staging repository ID (e.g., orgapacheparquet-1234)
+
+Options:
+  --rc <num>            RC number that passed the vote (default: auto-detect latest)
+  --allow-description-mismatch
+                        Bypass the staging-repo description check (for recovery scenarios)
+  --help                Show this help
+
+Before any destructive action, this script verifies that the staging repo
+belongs to org.apache.parquet, is in 'closed' state, contains
+${NEXUS_VERIFY_ARTIFACT_ID:-parquet-common}-<version>.pom, and has a
+description matching "Apache Parquet <version> RC<num>".
+
+The next development version is auto-computed by incrementing the patch
+version (e.g., 1.18.0 -> 1.18.1-SNAPSHOT).
+
+Environment variables:
+  DRY_RUN               Set to 0 for real execution (default: 1)
+  NEXUS_USERNAME        Apache Nexus username
+  NEXUS_PASSWORD        Apache Nexus password
+  SVN_USERNAME          SVN username for dist.apache.org
+  SVN_PASSWORD          SVN password
+  GITHUB_TOKEN          GitHub token for release creation
+
+Example:
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234 --rc 2
+EOF
+  exit "${1:-0}"
+}
+
+# ---------------------------------------------------------------------------
+# Parse arguments
+# ---------------------------------------------------------------------------
+version=""
+staging_repo_id=""
+rc_num=""
+allow_description_mismatch=0
+positional=()
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --rc)
+      if [[ -z "${2:-}" ]]; then
+        print_error "--rc requires a value"
+        usage 1
+      fi
+      rc_num="$2"
+      shift 2
+      ;;
+    --allow-description-mismatch)
+      allow_description_mismatch=1
+      shift
+      ;;
+    --help|-h)
+      usage 0
+      ;;
+    -*)
+      print_error "Unknown option: $1"
+      usage 1
+      ;;
+    *)
+      positional+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [[ ${#positional[@]} -lt 2 ]]; then
+  print_error "Expected 2 positional arguments (version, staging-repo-id), got ${#positional[@]}"
+  usage 1
+fi
+
+version="${positional[0]}"
+staging_repo_id="${positional[1]}"
+
+# ---------------------------------------------------------------------------
+# Validate inputs
+# ---------------------------------------------------------------------------
+step_summary "## Release Publication"
+step_summary ""
+
+if [[ ${DRY_RUN:-1} -eq 1 ]]; then
+  step_summary "> **DRY RUN** -- no changes will be made"
+  step_summary ""
+fi
+
+if ! validate_and_extract_version "${version}"; then
+  print_error "Invalid version format: '${version}'"
+  exit 1
+fi
+
+next_dev_version="${major}.${minor}.$(( patch + 1 ))"
+
+if ! [[ "${staging_repo_id}" =~ ^[a-zA-Z][a-zA-Z0-9._-]*$ ]]; then
+  print_error "Invalid staging repository ID: '${staging_repo_id}'. Expected alphanumeric with dots/hyphens (e.g., orgapacheparquet-1234)."
+  exit 1
+fi
+
+if [[ -z "${rc_num}" ]]; then
+  print_info "No RC number specified, auto-detecting latest RC for ${version}..."
+  if ! find_latest_rc_number "${version}"; then
+    exit 1
+  fi
+  rc_num="${latest_rc_number}"
+  print_info "Auto-detected latest RC: rc${rc_num}"
+else
+  if ! [[ "${rc_num}" =~ ^[0-9]+$ ]]; then
+    print_error "Invalid RC number: '${rc_num}'. Expected a non-negative integer."
+    exit 1
+  fi
+
+  if find_latest_rc_number "${version}" 2>/dev/null; then
+    if [[ "${rc_num}" -ne "${latest_rc_number}" ]]; then
+      print_error "RC${rc_num} is not the latest RC for ${version}. Latest is rc${latest_rc_number}."
+      print_error "Publishing an older RC is likely a mistake. If intentional, delete the newer RC tags first."
+      exit 1
+    fi
+  fi
+fi
+
+rc_tag="${TAG_PREFIX}${version}-rc${rc_num}"
+final_tag="${TAG_PREFIX}${version}"
+
+if ! git rev-parse "${rc_tag}" >/dev/null 2>&1; then
+  print_error "RC tag ${rc_tag} does not exist"
+  exit 1
+fi
+
+rc_commit=$(git rev-list -1 "${rc_tag}")
+current_commit=$(git rev-parse HEAD)
+
+if [[ "${current_commit}" != "${rc_commit}" ]]; then
+  print_error "Current HEAD (${current_commit}) does not match RC tag ${rc_tag} (${rc_commit})"
+  print_error "The release branch has commits beyond the voted RC. Either reset the branch or create a new RC."
+  exit 1
+fi
+
+if git rev-parse "${final_tag}" >/dev/null 2>&1; then
+  print_error "Final release tag ${final_tag} already exists"
+  exit 1
+fi
+
+step_summary "| Parameter | Value |"
+step_summary "| --- | --- |"
+step_summary "| Version | \`${version}\` |"
+step_summary "| RC tag | \`${rc_tag}\` |"
+step_summary "| Final tag | \`${final_tag}\` |"
+step_summary "| Staging repo | \`${staging_repo_id}\` |"
+step_summary "| Next dev version | \`${next_dev_version}-SNAPSHOT\` |"
+step_summary "| Commit | \`${rc_commit}\` |"
+
+# ---------------------------------------------------------------------------
+# Step 0: Verify staging repository before any destructive action
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Staging Repository Verification"
+
+if ! nexus_verify_staging_repo "${staging_repo_id}" "${version}" "${rc_num}" "${allow_description_mismatch}"; then
+  step_summary "Staging repository verification: **FAILED**"
+  exit 1
+fi
+step_summary "Staging repository \`${staging_repo_id}\` verified"
+
+# ---------------------------------------------------------------------------
+# Step 1: Move SVN artifacts from dist/dev to dist/release
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### SVN Promotion"
+
+svn_promote_rc_to_release "${version}" "${rc_num}"
+
+step_summary "Promoted \`${rc_tag}\` -> \`${final_tag}\` on dist.apache.org"
+
+# ---------------------------------------------------------------------------
+# Step 2: Clean up superseded patch releases on the same minor branch
+# ---------------------------------------------------------------------------
+# Policy: only patch releases of the *same* minor (parquet-${major}.${minor}.x)

Review Comment:
   Yes, I'm trying not to get to far away from what Parquet has done in the past without consensus. Previously we have just left pretty much everything up



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-5309639068

   > based on some experience elsewhere, I now realise it's important to to disable gha's on forked repos...not so much an issue for the manually triggered stuff compared to cron jobs, but still a good condition to get used to disabling things like release pipelines there
   
   See
   https://github.com/apache/parquet-java/pull/3548/changes#diff-f81cbb6af2733ed5e2176956d022b3777ee6c80a48336e81930016bdbfa555a4R54
   
   These shouldn't exist on any forks unless you also have the strict github name of "apache" :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197631647


##########
release/libs/_maven.sh:
##########
@@ -0,0 +1,104 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[[ -n "${_MAVEN_LOADED:-}" ]] && return 0 2>/dev/null || true
+_MAVEN_LOADED=1
+
+LIBS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_exec.sh"
+
+function _xml_escape {
+  local str="$1"
+  str="${str//&/&amp;}"
+  str="${str//</&lt;}"
+  str="${str//>/&gt;}"
+  str="${str//\"/&quot;}"
+  str="${str//\'/&apos;}"
+  echo "${str}"
+}
+
+function generate_maven_settings {
+  local settings_file="${1:-.release-settings.xml}"
+
+  if [[ -z "${NEXUS_USERNAME:-}" || -z "${NEXUS_PASSWORD:-}" ]]; then
+    print_warning "NEXUS_USERNAME or NEXUS_PASSWORD not set; Maven deploy may fail"
+  fi
+
+  local esc_username esc_password
+  esc_username=$(_xml_escape "${NEXUS_USERNAME:-}")
+  esc_password=$(_xml_escape "${NEXUS_PASSWORD:-}")
+
+  (
+    umask 077
+    cat > "${settings_file}" <<EOF
+<?xml version="1.0" encoding="UTF-8"?>
+<settings xmlns="http://maven.apache.org/SETTINGS/1.2.0"
+          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 https://maven.apache.org/xsd/settings-1.2.0.xsd">
+  <servers>
+    <server>
+      <id>apache.releases.https</id>
+      <username>${esc_username}</username>

Review Comment:
   This feels like a security issue, unlikely but someone could modify this or other steps and echo out the contents of this file, so let's switch to using env variables for maven



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197555567


##########
.github/workflows/release-prepare-rc.yml:
##########
@@ -0,0 +1,89 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Prepare RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number override (leave empty for auto-detect)'
+        required: false
+        type: string
+        default: ''
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+jobs:
+  prepare-rc:
+    name: Prepare Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: '11'
+
+      - name: Import GPG key
+        uses: crazy-max/ghaction-import-gpg@v6

Review Comment:
   This used to use some random repo's gh import gpg action, I don't think we need the dependency 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197660313


##########
release/libs/_version.sh:
##########
@@ -0,0 +1,136 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[[ -n "${_VERSION_LOADED:-}" ]] && return 0 2>/dev/null || true
+_VERSION_LOADED=1
+
+LIBS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+source "$LIBS_DIR/_constants.sh"
+source "$LIBS_DIR/_exec.sh"
+
+function validate_and_extract_version {
+  local version="$1"
+  if [[ ! ${version} =~ ^${VERSION_REGEX}$ ]]; then
+    return 1
+  fi
+  major="${BASH_REMATCH[1]}"
+  minor="${BASH_REMATCH[2]}"
+  patch="${BASH_REMATCH[3]}"
+  version_without_rc="${major}.${minor}.${patch}"
+  return 0
+}
+
+function validate_and_extract_git_tag_version {
+  local tag="$1"
+  if [[ ! ${tag} =~ ${VERSION_REGEX_GIT_TAG} ]]; then
+    return 1
+  fi
+  major="${BASH_REMATCH[1]}"
+  minor="${BASH_REMATCH[2]}"
+  patch="${BASH_REMATCH[3]}"
+  rc_number="${BASH_REMATCH[4]}"
+  version_without_rc="${major}.${minor}.${patch}"
+  return 0
+}
+
+function validate_and_extract_branch_version {
+  local branch="$1"
+  if [[ ! ${branch} =~ ${BRANCH_VERSION_REGEX} ]]; then
+    return 1
+  fi
+  major="${BASH_REMATCH[1]}"
+  minor="${BASH_REMATCH[2]}"
+  return 0
+}
+
+function _filter_rc_tags {

Review Comment:
   Another paranoia from me, stops us from finding a 1.9.0-rcFoo being considered



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3221613321


##########
release/bin/publish-release.sh:
##########
@@ -0,0 +1,408 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+LIBS_DIR="${SCRIPT_DIR}/../libs"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_log.sh"
+source "${LIBS_DIR}/_exec.sh"
+source "${LIBS_DIR}/_version.sh"
+source "${LIBS_DIR}/_nexus.sh"
+source "${LIBS_DIR}/_svn.sh"
+
+# ---------------------------------------------------------------------------
+# Usage
+# ---------------------------------------------------------------------------
+function usage {
+  cat <<EOF
+Usage: $0 <version> <staging-repo-id> [--rc <num>] [--allow-description-mismatch]
+
+Publish a release after the vote passes.
+
+Arguments:
+  version               Release version (e.g., 1.18.0)
+  staging-repo-id       Nexus staging repository ID (e.g., orgapacheparquet-1234)
+
+Options:
+  --rc <num>            RC number that passed the vote (default: auto-detect latest)
+  --allow-description-mismatch
+                        Bypass the staging-repo description check (for recovery scenarios)
+  --help                Show this help
+
+Before any destructive action, this script verifies that the staging repo
+belongs to org.apache.parquet, is in 'closed' state, contains
+${NEXUS_VERIFY_ARTIFACT_ID:-parquet-common}-<version>.pom, and has a
+description matching "Apache Parquet <version> RC<num>".
+
+The next development version is auto-computed by incrementing the patch
+version (e.g., 1.18.0 -> 1.18.1-SNAPSHOT).
+
+Environment variables:
+  DRY_RUN               Set to 0 for real execution (default: 1)
+  NEXUS_USERNAME        Apache Nexus username
+  NEXUS_PASSWORD        Apache Nexus password
+  SVN_USERNAME          SVN username for dist.apache.org
+  SVN_PASSWORD          SVN password
+  GITHUB_TOKEN          GitHub token for release creation
+
+Example:
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234
+  DRY_RUN=1 $0 1.18.0 orgapacheparquet-1234 --rc 2
+EOF
+  exit "${1:-0}"
+}
+
+# ---------------------------------------------------------------------------
+# Parse arguments
+# ---------------------------------------------------------------------------
+version=""
+staging_repo_id=""
+rc_num=""
+allow_description_mismatch=0
+positional=()
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --rc)
+      if [[ -z "${2:-}" ]]; then
+        print_error "--rc requires a value"
+        usage 1
+      fi
+      rc_num="$2"
+      shift 2
+      ;;
+    --allow-description-mismatch)
+      allow_description_mismatch=1
+      shift
+      ;;
+    --help|-h)
+      usage 0
+      ;;
+    -*)
+      print_error "Unknown option: $1"
+      usage 1
+      ;;
+    *)
+      positional+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [[ ${#positional[@]} -lt 2 ]]; then
+  print_error "Expected 2 positional arguments (version, staging-repo-id), got ${#positional[@]}"
+  usage 1
+fi
+
+version="${positional[0]}"
+staging_repo_id="${positional[1]}"
+
+# ---------------------------------------------------------------------------
+# Validate inputs
+# ---------------------------------------------------------------------------
+step_summary "## Release Publication"
+step_summary ""
+
+if [[ ${DRY_RUN:-1} -eq 1 ]]; then
+  step_summary "> **DRY RUN** -- no changes will be made"
+  step_summary ""
+fi
+
+if ! validate_and_extract_version "${version}"; then
+  print_error "Invalid version format: '${version}'"
+  exit 1
+fi
+
+next_dev_version="${major}.${minor}.$(( patch + 1 ))"
+
+if ! [[ "${staging_repo_id}" =~ ^[a-zA-Z][a-zA-Z0-9._-]*$ ]]; then
+  print_error "Invalid staging repository ID: '${staging_repo_id}'. Expected alphanumeric with dots/hyphens (e.g., orgapacheparquet-1234)."
+  exit 1
+fi
+
+if [[ -z "${rc_num}" ]]; then
+  print_info "No RC number specified, auto-detecting latest RC for ${version}..."
+  if ! find_latest_rc_number "${version}"; then
+    exit 1
+  fi
+  rc_num="${latest_rc_number}"
+  print_info "Auto-detected latest RC: rc${rc_num}"
+else
+  if ! [[ "${rc_num}" =~ ^[0-9]+$ ]]; then
+    print_error "Invalid RC number: '${rc_num}'. Expected a non-negative integer."
+    exit 1
+  fi
+
+  if find_latest_rc_number "${version}" 2>/dev/null; then
+    if [[ "${rc_num}" -ne "${latest_rc_number}" ]]; then
+      print_error "RC${rc_num} is not the latest RC for ${version}. Latest is rc${latest_rc_number}."
+      print_error "Publishing an older RC is likely a mistake. If intentional, delete the newer RC tags first."
+      exit 1
+    fi
+  fi
+fi
+
+rc_tag="${TAG_PREFIX}${version}-rc${rc_num}"
+final_tag="${TAG_PREFIX}${version}"
+
+if ! git rev-parse "${rc_tag}" >/dev/null 2>&1; then
+  print_error "RC tag ${rc_tag} does not exist"
+  exit 1
+fi
+
+rc_commit=$(git rev-list -1 "${rc_tag}")
+current_commit=$(git rev-parse HEAD)
+
+if [[ "${current_commit}" != "${rc_commit}" ]]; then
+  print_error "Current HEAD (${current_commit}) does not match RC tag ${rc_tag} (${rc_commit})"
+  print_error "The release branch has commits beyond the voted RC. Either reset the branch or create a new RC."
+  exit 1
+fi
+
+if git rev-parse "${final_tag}" >/dev/null 2>&1; then
+  print_error "Final release tag ${final_tag} already exists"
+  exit 1
+fi
+
+step_summary "| Parameter | Value |"
+step_summary "| --- | --- |"
+step_summary "| Version | \`${version}\` |"
+step_summary "| RC tag | \`${rc_tag}\` |"
+step_summary "| Final tag | \`${final_tag}\` |"
+step_summary "| Staging repo | \`${staging_repo_id}\` |"
+step_summary "| Next dev version | \`${next_dev_version}-SNAPSHOT\` |"
+step_summary "| Commit | \`${rc_commit}\` |"
+
+# ---------------------------------------------------------------------------
+# Step 0: Verify staging repository before any destructive action
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Staging Repository Verification"
+
+if ! nexus_verify_staging_repo "${staging_repo_id}" "${version}" "${rc_num}" "${allow_description_mismatch}"; then
+  step_summary "Staging repository verification: **FAILED**"
+  exit 1
+fi
+step_summary "Staging repository \`${staging_repo_id}\` verified"
+
+# ---------------------------------------------------------------------------
+# Step 1: Move SVN artifacts from dist/dev to dist/release
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### SVN Promotion"
+
+svn_promote_rc_to_release "${version}" "${rc_num}"
+
+step_summary "Promoted \`${rc_tag}\` -> \`${final_tag}\` on dist.apache.org"
+
+# ---------------------------------------------------------------------------
+# Step 2: Clean up superseded patch releases on the same minor branch
+# ---------------------------------------------------------------------------
+# Policy: only patch releases of the *same* minor (parquet-${major}.${minor}.x)

Review Comment:
   So heads up to reviewers, We currently do not have any instructions to remove old releases so this is new. Apache policy says we shouldn't keep *any* old releases here, they should just be in archives. Currently It looks like some Parquet releases were removed and some weren't. *This changes the behavior to only keep one patch per minor.*



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3350678830


##########
.github/workflows/ci-release-scripts.yml:
##########
@@ -0,0 +1,62 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Test Release Scripts
+
+on:
+  pull_request:
+    paths:
+      - 'release/**'
+  push:
+    branches:
+      - master
+    paths:
+      - 'release/**'
+
+jobs:
+  bats:
+    name: Release Script Unit Tests
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4

Review Comment:
   looking @ dependabot updated scripts elsewhere, the latest is
   ```
   actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
   ```
   



##########
.github/workflows/release-prepare-rc.yml:
##########
@@ -0,0 +1,94 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Prepare RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number override (leave empty for auto-detect)'
+        required: false
+        type: string
+        default: ''
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+concurrency:
+  group: release-${{ inputs.version }}
+  cancel-in-progress: false
+
+jobs:
+  prepare-rc:
+    if: github.repository_owner == 'apache'
+    name: Prepare Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@v4

Review Comment:
         uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0



##########
release/libs/_github.sh:
##########
@@ -0,0 +1,77 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[[ -n "${_GITHUB_LOADED:-}" ]] && return 0 2>/dev/null || true
+_GITHUB_LOADED=1
+
+LIBS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+source "${LIBS_DIR}/_log.sh"
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_exec.sh"
+
+function check_github_checks_passed() {
+  local commit_sha="$1"
+
+  print_info "Checking GitHub CI status for commit ${commit_sha}..."
+
+  if [[ ${DRY_RUN:-1} -eq 1 ]]; then
+    print_info "DRY_RUN is enabled, skipping GitHub check verification"
+    return 0
+  fi
+
+  if [[ -z "${GITHUB_TOKEN:-}" ]]; then
+    print_error "GITHUB_TOKEN is required to verify CI checks"
+    return 1
+  fi
+
+  local repo_info="${GITHUB_REPO}"
+
+  local num_incomplete
+  if ! num_incomplete=$(gh api "repos/${repo_info}/commits/${commit_sha}/check-runs" \
+    --jq '[.check_runs[] | select(.status != "completed")] | length'); then
+    print_error "Failed to fetch GitHub check runs for commit ${commit_sha}"
+    return 1
+  fi
+
+  if [[ ${num_incomplete} -ne 0 ]]; then
+    print_error "Found ${num_incomplete} still-running GitHub checks for commit ${commit_sha}"
+    gh api "repos/${repo_info}/commits/${commit_sha}/check-runs" \
+      --jq '.check_runs[] | select(.status != "completed") | "  - \(.name): \(.status)"' >&2
+    return 1
+  fi
+
+  local num_failed
+  if ! num_failed=$(gh api "repos/${repo_info}/commits/${commit_sha}/check-runs" \
+    --jq '[.check_runs[] | select(.conclusion != "success" and .conclusion != "skipped")] | length'); then
+    print_error "Failed to fetch GitHub check runs for commit ${commit_sha}"
+    return 1
+  fi
+
+  if [[ ${num_failed} -ne 0 ]]; then
+    print_error "Found ${num_failed} failed GitHub checks for commit ${commit_sha}"
+    gh api "repos/${repo_info}/commits/${commit_sha}/check-runs" \
+      --jq '.check_runs[] | select(.conclusion != "success" and .conclusion != "skipped") | "  - \(.name): \(.conclusion)"' >&2
+    return 1
+  fi
+
+  print_info "All GitHub checks passed for commit ${commit_sha}"

Review Comment:
   so you can't make a release from a broken build, but the checks are decoupled from the actual release process which is therefore nice and fast? slick.



##########
release/libs/_maven.sh:
##########
@@ -0,0 +1,87 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[[ -n "${_MAVEN_LOADED:-}" ]] && return 0 2>/dev/null || true
+_MAVEN_LOADED=1
+
+LIBS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_exec.sh"
+
+function generate_maven_settings {
+  local settings_file="${1:-.release-settings.xml}"
+
+  if [[ -z "${NEXUS_USERNAME:-}" || -z "${NEXUS_PASSWORD:-}" ]]; then
+    print_warning "NEXUS_USERNAME or NEXUS_PASSWORD not set; Maven deploy may fail"
+  fi
+
+  cat > "${settings_file}" <<'SETTINGS_EOF'
+<?xml version="1.0" encoding="UTF-8"?>

Review Comment:
   this is such a maven hack isn't it, it requiring storage of the credentials for publishing artifacts to be stored in plaintext. Hopefully the north koreans haven't added copying ~/.m2/settings.xml to their package manager worms. This release mech is *more secure* that all of us keeping it locally in the file



##########
.github/workflows/ci-release-scripts.yml:
##########
@@ -0,0 +1,62 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Test Release Scripts
+
+on:
+  pull_request:
+    paths:
+      - 'release/**'
+  push:
+    branches:
+      - master
+    paths:
+      - 'release/**'
+
+jobs:
+  bats:
+    name: Release Script Unit Tests
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0

Review Comment:
   add
   ```
           persist-credentials: false
   ```
   



##########
.github/workflows/release-cancel-rc.yml:
##########
@@ -0,0 +1,74 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Cancel RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number to cancel (e.g., 0)'
+        required: true
+        type: string
+      staging_repo_id:
+        description: 'Nexus staging repository ID to drop (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+jobs:
+  cancel-rc:
+    name: Cancel Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4

Review Comment:
   ```
   actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
   ```
   



##########
.github/workflows/release-cancel-rc.yml:
##########
@@ -0,0 +1,74 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Cancel RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number to cancel (e.g., 0)'
+        required: true
+        type: string
+      staging_repo_id:
+        description: 'Nexus staging repository ID to drop (e.g., orgapacheparquet-1234)'
+        required: true
+        type: string
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+jobs:
+  cancel-rc:
+    name: Cancel Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Install Subversion
+        run: sudo apt-get update && sudo apt-get install -y subversion
+
+      - name: Cancel Release Candidate
+        env:
+          DRY_RUN: ${{ inputs.dry_run && '1' || '0' }}

Review Comment:
   + env expansion is the way to defend against malicious inputs before they are executed in scripts



##########
release/bin/prepare-rc.sh:
##########
@@ -0,0 +1,392 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+LIBS_DIR="${SCRIPT_DIR}/../libs"
+
+source "${LIBS_DIR}/_constants.sh"
+source "${LIBS_DIR}/_log.sh"
+source "${LIBS_DIR}/_exec.sh"
+source "${LIBS_DIR}/_version.sh"
+source "${LIBS_DIR}/_github.sh"
+source "${LIBS_DIR}/_nexus.sh"
+source "${LIBS_DIR}/_maven.sh"
+source "${LIBS_DIR}/_svn.sh"
+
+trap 'rm -f .release-settings.xml' EXIT
+
+# ---------------------------------------------------------------------------
+# Usage
+# ---------------------------------------------------------------------------
+function usage {
+  cat <<EOF
+Usage: $0 <version> [OPTIONS]
+
+Prepare a release candidate for Apache Parquet Java.
+
+Arguments:
+  version               Release version (e.g., 1.18.0)
+
+Options:
+  --rc <num>            Override RC number (default: auto-detect)
+  --skip-branch-creation  Do not create the release branch
+  --help                Show this help
+
+Environment variables:
+  DRY_RUN               Set to 0 for real execution (default: 1)
+  NEXUS_USERNAME        Apache Nexus username
+  NEXUS_PASSWORD        Apache Nexus password
+  SVN_USERNAME          SVN username for dist.apache.org
+  SVN_PASSWORD          SVN password
+  GITHUB_TOKEN          GitHub token for CI checks and release creation
+
+Example:
+  DRY_RUN=1 $0 1.18.0
+  DRY_RUN=0 $0 1.18.0 --rc 2
+EOF
+  exit "${1:-0}"
+}
+
+# ---------------------------------------------------------------------------
+# Parse arguments
+# ---------------------------------------------------------------------------
+version=""
+rc_override=""
+skip_branch=false
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --rc)
+      if [[ -z "${2:-}" ]]; then
+        print_error "--rc requires a value"
+        usage 1
+      fi
+      rc_override="$2"
+      if [[ ! "${rc_override}" =~ ^[0-9]+$ ]]; then
+        print_error "--rc value must be a non-negative integer, got: '${rc_override}'"
+        exit 1
+      fi
+      shift 2
+      ;;
+    --skip-branch-creation)
+      skip_branch=true
+      shift
+      ;;
+    --help|-h)
+      usage 0
+      ;;
+    -*)
+      print_error "Unknown option: $1"
+      usage 1
+      ;;
+    *)
+      if [[ -z "${version}" ]]; then
+        version="$1"
+      else
+        print_error "Unexpected argument: $1"
+        usage 1
+      fi
+      shift
+      ;;
+  esac
+done
+
+if [[ -z "${version}" ]]; then
+  print_error "Version is required"
+  usage 1
+fi
+
+# ---------------------------------------------------------------------------
+# Step 0: Validate inputs
+# ---------------------------------------------------------------------------
+step_summary "## Release Candidate Preparation"
+step_summary ""
+
+if [[ ${DRY_RUN:-1} -eq 1 ]]; then
+  step_summary "> **DRY RUN** -- no changes will be made"
+  step_summary ""
+fi
+
+if ! validate_and_extract_version "${version}"; then
+  print_error "Invalid version format: '${version}'. Expected: X.Y.Z"
+  exit 1
+fi
+
+step_summary "| Parameter | Value |"
+step_summary "| --- | --- |"
+step_summary "| Version | \`${version}\` |"
+
+# Check prerequisites
+if ! command -v gpg &>/dev/null; then
+  print_warning "gpg not found -- GPG signing will fail"
+fi
+if ! command -v svn &>/dev/null; then
+  print_warning "svn not found -- SVN staging will fail"
+fi
+if [[ ! -x ./mvnw ]]; then
+  print_error "mvnw not found in current directory. Run from the repo root."
+  exit 1
+fi
+
+# ---------------------------------------------------------------------------
+# Step 1: Create release branch (idempotent)
+# ---------------------------------------------------------------------------
+release_branch="${BRANCH_PREFIX}${major}.${minor}.x"
+step_summary "| Release branch | \`${release_branch}\` |"
+
+if [[ "${skip_branch}" == "true" ]]; then
+  print_info "Skipping branch creation (--skip-branch-creation)"
+elif git show-ref --verify --quiet "refs/remotes/origin/${release_branch}" 2>/dev/null; then
+  print_info "Release branch ${release_branch} already exists, skipping creation"
+else
+  print_info "Creating release branch ${release_branch} from master..."
+  exec_process git branch "${release_branch}" origin/master
+  exec_process git push origin "${release_branch}" --set-upstream
+  step_summary ""
+  step_summary "Created release branch \`${release_branch}\`"
+fi
+
+# Switch to the release branch
+if [[ "$(git branch --show-current)" != "${release_branch}" ]]; then
+  print_info "Switching to ${release_branch}..."
+  exec_process git checkout "${release_branch}"
+fi
+
+# ---------------------------------------------------------------------------
+# Step 2: Auto-detect RC number
+# ---------------------------------------------------------------------------
+if [[ -n "${rc_override}" ]]; then
+  rc_number="${rc_override}"
+  print_info "Using RC override: rc${rc_number}"
+else
+  find_next_rc_number "${version}"
+  print_info "Auto-detected next RC: rc${rc_number}"
+fi
+
+rc_tag="${TAG_PREFIX}${version}-rc${rc_number}"
+step_summary "| RC number | \`${rc_number}\` |"
+step_summary "| RC tag | \`${rc_tag}\` |"
+
+# Check if tag already exists
+if git rev-parse "${rc_tag}" >/dev/null 2>&1; then
+  print_error "Tag ${rc_tag} already exists. Use --rc to specify a different RC number."
+  exit 1
+fi
+
+# ---------------------------------------------------------------------------
+# Step 3: Verify CI checks
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### CI Verification"
+
+current_commit=$(git rev-parse HEAD)
+step_summary "| Verified commit | \`${current_commit}\` |"
+
+if ! check_github_checks_passed "${current_commit}"; then
+  print_error "CI checks are not passing. Fix CI before creating an RC."
+  step_summary "CI checks: **FAILED**"
+  exit 1
+fi
+step_summary "CI checks on \`${current_commit}\`: **PASSED**"
+step_summary ""
+step_summary "> **Note:** the RC tag is placed on a child commit (\"Set"
+step_summary "> version to ${version} for release\") that is created in"
+step_summary "> the next step. That commit is not directly CI-verified;"
+step_summary "> it is a mechanical version bump on top of the verified"
+step_summary "> commit above."
+
+# ---------------------------------------------------------------------------
+# Step 4: Set POM versions (only on rc0 or if version doesn't match)
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Version Update"
+
+current_pom_version=$(get_current_pom_version || echo "unknown")
+print_info "Current POM version: ${current_pom_version}"
+
+if [[ "${current_pom_version}" == "${version}" ]]; then
+  print_info "POM version already set to ${version}, skipping version update"
+  step_summary "POM version already at \`${version}\`, no update needed"
+else
+  print_info "Setting POM version to ${version}..."
+  set_pom_version "${version}"
+  step_summary "Updated POM version: \`${current_pom_version}\` -> \`${version}\`"
+
+  # Commit version changes
+  exec_process git add -A
+  exec_process git commit -m "Set version to ${version} for release"
+fi
+
+# ---------------------------------------------------------------------------
+# Step 5: Create RC tag and push
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Tag and Push"
+
+exec_process git tag -a "${rc_tag}" -m "Apache Parquet ${version} RC${rc_number}"
+exec_process git push origin "${release_branch}"
+exec_process git push origin "${rc_tag}"
+
+tag_commit=$(git rev-parse HEAD)
+step_summary "Created tag \`${rc_tag}\` at \`${tag_commit}\`"
+
+# ---------------------------------------------------------------------------
+# Step 6: Deploy to Nexus
+# ---------------------------------------------------------------------------
+step_summary ""
+step_summary "### Nexus Deployment"
+
+settings_file=".release-settings.xml"
+generate_maven_settings "${settings_file}"
+
+maven_deploy "${settings_file}"
+
+step_summary "Deployed artifacts to Apache Nexus staging"
+
+# Find and close the staging repo. Verify it actually contains the
+# artifacts we just deployed before closing -- another concurrent run

Review Comment:
   I've hit some bizarre stuff with behind-VPN releases where a single  docker container release build ended up pushing up artifacts to multiple staging repos -nexus splits repos by source IP Address for security, and and if routing is inconsistent the release breaks.
   
   hopefully GHA hosting will be good, otherwise more artifact auditing is needed (i.e. everything)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197515919


##########
.github/workflows/ci-release-scripts.yml:
##########
@@ -0,0 +1,59 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Test Release Scripts
+
+on:
+  pull_request:
+    paths:
+      - 'release/**'
+  push:
+    branches:
+      - master
+    paths:
+      - 'release/**'
+
+jobs:

Review Comment:
   We can rip out all of these tests if folks don't find them useful, it's just a bunch of Bash Unit Testing to try to make sure at least our helper functions do what they are supposed to. They use mocked output though so they have limited utility for testing the real true pathway.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-4543813271

   I've just discovered that the apache tooling project is building a service "Apache Trusted Releases"
   
   https://github.com/apache/tooling-trusted-releases
   you can already experiment with this on a project you're on the PMC on
   https://release-test.apache.org/
   
   This is more ambitious as they're trying to automate the entire release process in a way that somehow builds the release from the repo, so avoids the "malicious release manager"  threat.
   
   It can do audits of the artifacts, the usual vote-time due-diligence, but it also permits a project to have fully automated publishing : https://release-test.apache.org/docs/trusted-publishing
   
   Whatever is done here should be done with this in mind
   -ATR can be used to audit releases today
   -Trusted Releases should be a goal of projects, not just for security but for the ability to push out new releases fast. I think there's still that need for some human intervention (it's that full automation which has burned NPM packaging), but the speed of supply chain issues means that projects need to be set up for a rapid release and a momentum of something every week, somehow


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197535377


##########
.github/workflows/release-prepare-rc.yml:
##########
@@ -0,0 +1,89 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Release - Prepare RC
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version (e.g., 1.18.0)'
+        required: true
+        type: string
+      rc_number:
+        description: 'RC number override (leave empty for auto-detect)'
+        required: false
+        type: string
+        default: ''
+      dry_run:
+        description: 'Dry run mode (no actual changes)'
+        required: false
+        type: boolean
+        default: true
+
+jobs:
+  prepare-rc:
+    name: Prepare Release Candidate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up JDK 11
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: '11'
+
+      - name: Import GPG key
+        uses: crazy-max/ghaction-import-gpg@v6

Review Comment:
   This is wrong, we want to use the infra key - will fix



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#discussion_r3197603666


##########
release/libs/_exec.sh:
##########
@@ -0,0 +1,92 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[[ -n "${_EXEC_LOADED:-}" ]] && return 0 2>/dev/null || true
+_EXEC_LOADED=1
+
+LIBS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+source "$LIBS_DIR/_constants.sh"
+source "$LIBS_DIR/_log.sh"
+
+function _redact_secrets {

Review Comment:
   This is me being paranoid for local runs, Github will do this automatically but incase someone wants to run this locally and copies and pastes, this will protect their secrets.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-5258544236

   Thanks @RussellSpitzer for working on this 🚀  Thanks @wgtmac and @steveloughran for the review 🙌 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-5307915764

   based on some experience elsewhere, I now realise it's important to to disable gha's on forked repos...not so much an issue for the manually triggered stuff compared to cron jobs, but still a good condition to get used to disabling things like release pipelines there


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3547: Add semi-automated release pipeline for Apache Parquet Java [parquet-java]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #3548:
URL: https://github.com/apache/parquet-java/pull/3548#issuecomment-5258757975

   Infra ticket - https://issues.apache.org/jira/browse/INFRA-28245


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org