You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "sgilmore10 (via GitHub)" <gi...@apache.org> on 2024/04/10 18:52:02 UTC
[PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
sgilmore10 opened a new pull request, #41131:
URL: https://github.com/apache/arrow/pull/41131
### Rationale for this change
As per @kou's [suggestion](https://github.com/apache/arrow/pull/40956#discussion_r1556447060) in #40956, we should create unique git tags (e.g. `apache-arrow-{MAJOR}.{MINOR}.{VERSION}-rc{RC_NUM}`) instead re-using the same git tag (`apache-arrow-{MAJOR}.{MINOR}.{VERSION}`) for each release candidate. The official release candidate tag (`apache-arrow-{MAJOR}.{MINOR}.{VERSION}`) should be created **only** after a release candidate is voted on and accepted. This "official" release tag should point to the same object in the git database as the accepted release candidate tag.
The new release workflow could look like the following:
> 1. Create a apache-arrow-X.Y.Z-rc0 tag for X.Y.Z RC0
> 2. (Found a problem for X.Y.Z RC0)
> 3. Create a apache-arrow-X.Y.Z-rc1 tag for X.Y.Z RC1
> 4. Vote
> 5. Passed
> 6. Create a apache-arrow-X.Y.Z tag from apache-arrow-X.Y.Z-rc1 ike apache/arrow-adbc and apache/arrow-flight-sql-postgresql do
See @kou's [comment](https://github.com/apache/arrow/pull/40956#discussion_r1556447060) for more details.
### What changes are included in this PR?
1. Updated `dev/release/01-prepare.sh` to create release-candidate-specific git tags (e.g. `apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}`).
2. Updated scripts in `dev/release` to use the new git tag name.
3. Added GitHub Workflow file `publish_release_candidate.yml`. This workflow is triggered when a release candidate git tag is pushed and creates a Prerelease GitHub Release.
4. Added logic to `dev/release/02-post-binary.sh` to create and push the release git tag (i.e. `apache-arrow-{MAJOR}.{MINOR}.{PATCH}`).
5. Added GitHub Workflow `publish_release.yml`. This workflow is triggered when the release tag is pushed and creates a GitHub Release for the approved release (i.e. the voted upon release).
6. Added `dev/release/post-16-delete-release-candidates.sh` to delete the release candidate git tags and their associated GitHub Releases.
7. Updated `docs/developers/release.rst` with the new steps.
### Are these changes tested?
1. We were not able to verify the changes made to the scripts in `dev/release`. Any suggestions on how we can verify these scripts would be much appreciated :)
2. We did test the new GitHub Workflows (`publish_release_candidate.yml` and `publish_release.yml`) work as intended by pushing git tags to [`mathworks/arrow`](https://github.com/mathworks/arrow).
### Are there any user-facing changes?
No.
### Open Questions
1. We noticed that [apache/arrow-flight-sql-postgresql](https://github.com/apache/arrow-flight-sql-postgresql/releases) does **not** delete the release candidate Prereleases from their GitHub Releases area. Should we be doing the same? Or would it be preferable to just delete the the release candidates **without** deleting the release candidate tags.
2. We're not that familiar with ruby, so we're not sure if the changes we made to `dev/release/02-source-test.rb` make sense.
### Future Directions
1. Continue working on #40956
2. Add logic to auto-sign release artifacts in GitHub Actions Workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2137600122
Hi @assignUser,
I just wanted get your thoughts about moving forward with this PR after [the jira ticket](https://issues.apache.org/jira/browse/INFRA-25708) is resolved.
My understanding was that we can't merge this PR until we add the automated signing logic to the `release_candidate.yml` workflow. My original thought process was that we would need to wait to do this work until [INFRA-25708](https://issues.apache.org/jira/browse/INFRA-25708) is resolved. However, I'm wondering if it makes sense to start that work now so that we can merge this PR in time for the next release. If I do add the automated signing logic, I could either make a separate jira ticket or add a comment to the existing one.
What do you think?
Best,
Sarah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1567398151
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
+ echo "Release Notes URL: ${release_notes}"
+ echo "RELEASE_NOTES=${release_notes}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --verify-tag \
+ --title "${RELEASE_TITLE}" \
+ --notes "Release Notes: ${RELEASE_NOTES}"
Review Comment:
Will do!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561453621
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Get Latest Release Candidate's Git Tag
+ run: |
+ rc_tag=$(gh release list --repo ${{github.repository}} | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "ARROW_RELEASE_CANDIDATE_TAG=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title=$(gh release view ${ARROW_RELEASE_CANDIDATE_TAG} --repo ${{github.repository}} --json name --jq .name | \
+ sed -E -e 's/ RC[0-9]+$//')
+ echo "Release Title: ${title}"
+ echo "ARROW_RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes URL
+ run: |
+ version=${GITHUB_REF_NAME#apache-arrow-}
+ release_note_url="https://arrow.apache.org/release/${version}.html"
+ echo "Release Notes URL: ${release_note_url}"
+ echo "ARROW_RELEASE_NOTES_URL=${release_note_url}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --title "${ARROW_RELEASE_TITLE}" \
+ --verify-tag \
+ --notes "Release Notes: ${ARROW_RELEASE_NOTES_URL}" \
+ --repo ${{github.repository}}
Review Comment:
Marking as resolved because I will add this step in #40956
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1605047684
##########
.github/workflows/dev.yml:
##########
@@ -124,6 +124,7 @@ jobs:
- name: Run Release Test
env:
ARROW_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Review Comment:
According to the documentation [here](https://docs.github.com/en/actions/using-workflows/using-github-cli-in-workflows), you have to set an environment variable called `GH_TOKEN` in order to use GitHub CLI in a workflow.
However, setting `GH_TOKEN` may not longer be necessary in this step. At some point `ci/scripts/release_test.sh` was indirectly calling `gh release download` via `dev/release/02-source.sh`. However, I have since changed `dev/release/02-source.sh` to only call `gh release download` if the environment variable `SOURCE_DOWNLOAD` is greater than `0` (I believe @kou suggested this change).
It looks like `SOURCE_DOWNLOAD` is set `0` in `dev/release/02-source-test.rb` on line [39](https://github.com/apache/arrow/blob/068b9eccd46beb7516283ebc3a784b5866c7bcca/dev/release/02-source-test.rb#L39), which should make setting `GH_TOKEN` unnecessary. I'll try removing this line from `.github/workflows/dev.yml`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2118208258
Ignore my previous commit! I'm going to revert. I meant to temporarily delete all workflows except for `release.yml` and `release_candidate.yml` on a mathworks/arrow branch so that I could test the updated release workflows without triggering the other ones on push! So sorry!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2052271600
I was about to open an issue for INFRA to look over the workflow when I noticed that this PR also contains the changes from #40956 (+ some new changes on the workflows) is this intentional or do we want to move those changes back out of this PR? I just need clarity which PR I send to INFRA ^^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2116320972
@raulcd Could you review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2057147822
> Nice, great progress!
>
> > it made more sense to first add "generic" release candidate and release workflows
>
> Oh yeah that makes sense, though I do think we should also add the official tarball + signature to the releases. Looking at the release scripts the only change to make would be to upload the tarball to svn before pushing the (rc) tag, which triggers this workflow where we could then pull from dist.a.o .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1566296464
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
Based on the discussion, it seems like we should not trigger the release and release candidate workflows when pushing specific branches. Marking as resolved.
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
Review Comment:
Based on the discussion, it seems like we should not trigger the release and release candidate workflows when pushing specific branches. Marking as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1568140270
##########
dev/release/08-matlab-upload.sh:
##########
Review Comment:
Ah, sorry.
We should run `07-binary-verify.sh` after `08-matlab-upload.sh` to verify MLTBX files by `07-binary-verify.sh` (in the future). (We can add it to `verify-release-candidate.sh` later.)
Could you swap order of them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1568893669
##########
dev/release/02-source.sh:
##########
@@ -25,6 +25,7 @@ set -eu
: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
: ${SOURCE_PR:=${SOURCE_DEFAULT}}
: ${SOURCE_VOTE:=${SOURCE_DEFAULT}}
+: ${SOURCE_DOWNLOAD:=${SOURCE_DEFAULT}}
Review Comment:
Will do!
##########
dev/release/08-matlab-upload.sh:
##########
Review Comment:
Will do!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2057526604
> > +1 but we should wait for a review from @raulcd before we merge this. (@raulcd may review this after 16.0.0 release.)
> > I want to move the process that generates `apache-arrow-X.Y.Z.tar.gz` from `02-source.sh` to `.github/workflows/release_candidate.yml` as a separated task. Something like:
> > ```diff
> > diff --git a/dev/release/02-source.sh b/dev/release/02-source.sh
> > index 27ca67a76e..874c1955f8 100755
> > --- a/dev/release/02-source.sh
> > +++ b/dev/release/02-source.sh
> > @@ -57,33 +57,12 @@ echo "Using commit $release_hash"
> >
> > tarball=apache-arrow-${version}.tar.gz
> >
> > -rm -rf ${tag}
> > -# be conservative and use the release hash, even though git produces the same
> > -# archive (identical hashes) using the scm tag
> > -(cd "${SOURCE_TOP_DIR}" && \
> > - git archive ${release_hash} --prefix ${tag}/) | \
> > - tar xf -
> > -
> > -# Resolve all hard and symbolic links.
> > -# If we change this, we must change ArrowSources.archive in
> > -# dev/archery/archery/utils/source.py too.
> > -rm -rf ${tag}.tmp
> > -mv ${tag} ${tag}.tmp
> > -cp -R -L ${tag}.tmp ${tag}
> > -rm -rf ${tag}.tmp
> > -
> > -# Create a dummy .git/ directory to download the source files from GitHub with Source Link in C#.
> > -dummy_git=${tag}/csharp/dummy.git
> > -mkdir ${dummy_git}
> > -pushd ${dummy_git}
> > -echo ${release_hash} > HEAD
> > -echo '[remote "origin"] url = https://github.com/apache/arrow.git' >> config
> > -mkdir objects refs
> > -popd
> > -
> > -# Create new tarball from modified source directory
> > -tar czf ${tarball} ${tag}
> > -rm -rf ${tag}
> > +rm -f ${tarball}
> > +gh release download \
> > + ${tag} \
> > + --repo apache/arrow \
> > + --dir . \
> > + --pattern "${tarball}"
> >
> > if [ ${SOURCE_RAT} -gt 0 ]; then
> > "${SOURCE_DIR}/run-rat.sh" ${tarball}
> > ```
>
> This makes sense to me. However, I think it will require re-ordering the scripts to ensure we push the RC tag _**after**_ the crossbow job containing all build artifacts - specifically the MLTBX file - is finished. Otherwise, we won't be able to attach the MLTBX file - and potentially other build artifacts - as assets to the release candidate. What do you think?
On second thought, we could possibly write a "upload_release_candidate_assets" GitHub Actions that is triggered via [`gh workflow run`](https://cli.github.com/manual/gh_workflow_run) and is only called after the crossbow jobs are finished. I imagine the new release workflow would look something like the following:
```bash
# Push the release candidate tag - will create the GitHub Release
git push -u apache apache-arrow-<version>-rc<rc-number>
...
# Submit binary tasks using crossbow, the command will output the crossbow build id
dev/release/03-binary-submit.sh <version> <rc-number>
# Wait for the crossbow jobs to finish
archery crossbow status <crossbow-build-id>
# Extracts build artifacts from crossbow job (using 04-binary-download) and adds them as assets to the Release Candidate in apache/arrow's GitHub Release area
gh workflow run upload_release_candidate_assets.yml -f version=<version> rc=<rc-number>
...
```
Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1563738139
##########
docs/source/developers/release.rst:
##########
@@ -325,6 +326,18 @@ Be sure to go through on the following checklist:
Add relevant release data for Arrow to `Apache reporter <https://reporter.apache.org/addrelease.html?arrow>`_.
+.. dropdown:: Push release tag and create GitHub Release
+ :animate: fade-in-slide-down
+ :class-title: sd-fs-5
+ :class-container: sd-shadow-md
+
+ .. code-block:: Bash
+
+ A committer must push the release tag to GitHub:
Review Comment:
```suggestion
A committer must push the release tag to GitHub:
.. code-block:: Bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2148582319
Hi @kou,
I think we still need to wait on INFRA to approve [INFRA-25708](https://issues.apache.org/jira/browse/INFRA-25708) before we can merge this PR.
To clarify, do you think we could ask INFRA for a signing key for *just* the source tarballs? I was under the impression INFRA would not give out a signing key to use only for signing the source archive, but I may be mistaken.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2077726864
I have updated the issue to let INFRA know that this is now ready for their review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1605056687
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
That's a good point. Instead of setting the release notes here, I could add a new bash script that uses `gh release edit` to update the Release Notes only after the website is updated. Would that be preferable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1605067583
##########
docs/source/developers/release.rst:
##########
@@ -200,6 +200,8 @@ Build source and binaries and submit them
.. code-block::
# Build the source release tarball and create Pull Request with verification tasks
+ #
+ # Note that you need to have GitHub CLI installed to run this script.
Review Comment:
Will do!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561428622
##########
dev/release/post-02-binary.sh:
##########
@@ -108,3 +108,14 @@ docker_run \
VERBOSE=${VERBOSE:-no} \
VERSION=${version} \
YUM_TARGETS=$(IFS=,; echo "${yum_targets[*]}")
+
+# Create the release tag and trigger the Publish Release workflow.
+release_candidate_tag=apache-arrow-${version}-rc${num}
+release_tag=apache-arrow-${version}
+git tag -a ${release_tag} ${release_candidate_tag}^{} -m "[Release] Apache Arrow Release ${version}"
+git push origin ${release_tag}
+
+# Wait for the Publish Release workflow to finish.
+sleep 2s
+workflow_id=$(gh run list --repo apache/arrow --workflow "Publish Release" | cut -f7 | head -n1)
+gh run watch ${workflow_id} --repo apache/arrow
Review Comment:
Will do!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561882592
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+ tags:
+ # Trigger workflow when a tag whose name matches either
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" or pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
Review Comment:
How about creating two jobs for release and release candidate instead of putting `if` to each step?
```yaml
jobs:
publish-candidate:
if: |
contains(${GITHUB_REF_NAME}, 'rc')
...
publish:
if: |
!contains(${GITHUB_REF_NAME}, 'rc')
...
```
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
Review Comment:
Hmm. I think that we don't need run this workflow for branches. We need to run only for tags because this workflow creates a GitHub Release and it's associated with a Git tag.
@assignUser What do you think about this?
##########
dev/release/post-01-tag.sh:
##########
@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -o pipefail
+
+if [ "$#" -ne 2 ]; then
+ echo "Usage: $0 <version> <rc-num>"
+ exit
+fi
+
+# Create the release tag and trigger the Publish Release workflow.
+release_candidate_tag=apache-arrow-${version}-rc${num}
+release_tag=apache-arrow-${version}
+git tag -a ${release_tag} ${release_candidate_tag}^{} -m "[Release] Apache Arrow Release ${version}"
+git push origin ${release_tag}
+
+# Wait for the Publish Release workflow to finish.
+sleep 2s
+workflow_id=$(gh run list --repo apache/arrow --workflow "Publish Release" | cut -f7 | head -n1)
+gh run watch ${workflow_id} --repo apache/arrow
Review Comment:
We don't need this because we don't have following tasks that depend on the GitHub Release.
(If we remove this, we don't need `gh` for this.)
##########
dev/release/post-01-tag.sh:
##########
@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -o pipefail
+
+if [ "$#" -ne 2 ]; then
+ echo "Usage: $0 <version> <rc-num>"
+ exit
+fi
+
+# Create the release tag and trigger the Publish Release workflow.
+release_candidate_tag=apache-arrow-${version}-rc${num}
+release_tag=apache-arrow-${version}
+git tag -a ${release_tag} ${release_candidate_tag}^{} -m "[Release] Apache Arrow Release ${version}"
+git push origin ${release_tag}
Review Comment:
Could you use `apache` remote? We use it for apache/arrow and `origin` for `${your_github_account}/arrow`. ( https://arrow.apache.org/docs/developers/release.html#creating-a-release-candidate also uses `apache` remote to push a tag.)
```suggestion
git push apache ${release_tag}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561929955
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
Review Comment:
I think that a Git tag doesn't associated any branch. So `apache-arrow-X.Y.Z-rcN` tag on the `main` branch nor `apache-arrow-X.Y.Z-rcN` tag on the `release-X-Y-Z` branch don't exist.
> Btw I don't remember where the `release-*` branches come from? Are they created after the vote? What is their purpose compared to `maint-*`?
I think:
* `release-*` are for preparing `apache-arrow-X.Y.Z` tags
* `maint-X.Y.x` are for preparing `release-X.Y.Z` branches
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2152696891
> The MATLAB check crashed during the `Run Tests` step on Windows after I rebased on main to deal with merge conflicts. I'm fairly certain that this crash is unrelated to the changes in this PR because the same crash occurred in the mathworks/arrow fork after syncing the branch to be up to date with apache/arrow: https://github.com/mathworks/arrow/actions/runs/9401909013/job/25894734169
I've created issue for this crash: #42015
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2163687671
The [INFRA ticket](https://issues.apache.org/jira/browse/INFRA-25708) is now closed and I believe this PR is ready to merge. @assignUser, do you have any more flags? I think you just requested changes so we knew not to merge this PR until the Jira ticket was closed.
Otherwise, unless anyone has any other flags, I think I can merge this.
Also a big thank you to everyone who helped me!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 merged PR #41131:
URL: https://github.com/apache/arrow/pull/41131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1606078799
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
yeah! we can do that instead of adding a link that is broken. We can also add a comment on the documentation that after merging the PR created on `post-04-website.sh` we should run the new script.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1607199821
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
Just verified the new script `dev/release/post-05-update-gh-release-notes.sh` works as expected by using it to update the[ release notes of a release in the `mathworks/arrow` GitHub Releases area](https://github.com/mathworks/arrow/releases/tag/test-apache-arrow-16.0.0).
FYI, I'm going to delete that release in a day or so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561935341
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+ tags:
+ # Trigger workflow when a tag whose name matches either
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" or pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
Review Comment:
Sounds good to me. There's some similarity between the two jobs, but I suspect they'll start to diverge more as more steps are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2052314782
Just a quick update. I tested the new `release_candidate.yml` and `release.yml` workflows on the `mathworks/arrow` repository by pushing the tags `test-apache-arrow-16.0.0-rc0` and `test-apache-arrow-16.0.0` to `mathworks/arrow`. It's worth mentioning I modified the workflow files to expect the tag names to start with the prefix `test` (I didn't want to push tags whose names matched the patterns `apache-arrow-X.Y.Z` and `apache-arrow-X.Y.Z-rcN`).
Both the [release_candidate](https://github.com/mathworks/arrow/actions/runs/8666318372) and the [release](https://github.com/mathworks/arrow/actions/runs/8666329252/job/23766874707) workflows passed. You can view the generated releases in [mathworks/arrow's GitHub Releases area](https://github.com/mathworks/arrow/releases).
I'll delete those releases once this PR is done. For now, I'll leave them up to serve as a proof of concept - unless you prefer I delete asap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1563042040
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
Review Comment:
I just realized I completely misread Kou's comment about git tag's not being associated specific branches. Sorry about that! You can disregard my previous comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2052391116
> I was about to open an issue for INFRA to look over the workflow when I noticed that this PR also contains the changes from #40956 (+ some new changes on the workflows) is this intentional or do we want to move those changes back out of this PR? I just need clarity which PR I send to INFRA ^^
I think the plan is to first submit this PR and then rebase #40956 once this PR is merged. While working on #40956, I realized it made more sense to first add "generic" release candidate and release workflows before adding the MATLAB specific logic. Once this PR is merged, I'll go back to working on #40956. Sorry for the confusion!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1566707800
##########
dev/release/02-source.sh:
##########
@@ -56,35 +55,20 @@ fi
echo "Using commit $release_hash"
-tarball=${tag}.tar.gz
-
-rm -rf ${tag}
-# be conservative and use the release hash, even though git produces the same
-# archive (identical hashes) using the scm tag
-(cd "${SOURCE_TOP_DIR}" && \
- git archive ${release_hash} --prefix ${tag}/) | \
- tar xf -
-
-# Resolve all hard and symbolic links.
-# If we change this, we must change ArrowSources.archive in
-# dev/archery/archery/utils/source.py too.
-rm -rf ${tag}.tmp
-mv ${tag} ${tag}.tmp
-cp -R -L ${tag}.tmp ${tag}
-rm -rf ${tag}.tmp
-
-# Create a dummy .git/ directory to download the source files from GitHub with Source Link in C#.
-dummy_git=${tag}/csharp/dummy.git
-mkdir ${dummy_git}
-pushd ${dummy_git}
-echo ${release_hash} > HEAD
-echo '[remote "origin"] url = https://github.com/apache/arrow.git' >> config
-mkdir objects refs
-popd
-
-# Create new tarball from modified source directory
-tar czf ${tarball} ${tag}
-rm -rf ${tag}
+tarball=apache-arrow-${version}.tar.gz
+
+rm -f ${tarball}
+
+if [ ${TEST_RELEASE_SCRIPT} -gt 0 ]; then
+ # Assume calling from arrow's source directory
+ "${SOURCE_DIR}/utils-create-release-tarball.sh" ${version} ${rc}
+else
+ gh release download \
+ ${tag} \
+ --repo apache/arrow \
+ --dir . \
+ --pattern "${tarball}"
+fi
Review Comment:
How about using `apache-arrow-X.Y.Z.tar.gz` in local when `SOURCE_DOWNLOAD=0`?
```suggestion
if [ ${SOURCE_DOWNLOAD} -gt 0 ]; then
rm -f ${tarball}
gh release download \
${tag} \
--repo apache/arrow \
--dir . \
--pattern "${tarball}"
fi
```
##########
dev/release/utils-create-release-tarball.sh:
##########
@@ -0,0 +1,62 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
+
+if [ "$#" -ne 2 ]; then
+ echo "Usage: $0 <version> <rc-num>"
+ exit
+fi
+
+version=$1
+rc=$2
+
+tag=apache-arrow-${version}-rc${rc}
+tarball=apache-arrow-${version}.tar.gz
+
+: ${release_hash:=$(git rev-list --max-count=1 ${tag})}
+
+rm -rf ${tag}
+
+# be conservative and use the release hash, even though git produces the same
+# archive (identical hashes) using the scm tag
+(cd "${SOURCE_TOP_DIR}" && \
+ git archive ${release_hash} --prefix ${tag}/) | \
Review Comment:
```suggestion
git archive ${release_hash} --prefix ${tag}/) | \
```
##########
docs/source/developers/release.rst:
##########
@@ -325,6 +326,19 @@ Be sure to go through on the following checklist:
Add relevant release data for Arrow to `Apache reporter <https://reporter.apache.org/addrelease.html?arrow>`_.
+.. dropdown:: Push release tag and create GitHub Release
+ :animate: fade-in-slide-down
+ :class-title: sd-fs-5
+ :class-container: sd-shadow-md
+
+ A committer must push the release tag to GitHub:
+
+ .. code-block:: Bash
+
+
Review Comment:
```suggestion
```
##########
dev/release/02-source-test.rb:
##########
@@ -36,12 +37,13 @@ def source(*targets)
env = {
"SOURCE_DEFAULT" => "0",
"release_hash" => @current_commit,
+ "TEST_RELEASE_SCRIPT" => "1",
}
targets.each do |target|
env["SOURCE_#{target}"] = "1"
end
output = sh(env, @script, @release_version, "0")
Review Comment:
```suggestion
sh(env, File.expand_path("dev/release/utils-create-release-tarball.sh", @release_version, "0")
output = sh(env, @script, @release_version, "0")
```
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
+ echo "Release Notes URL: ${release_notes}"
+ echo "RELEASE_NOTES=${release_notes}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --verify-tag \
+ --title "${RELEASE_TITLE}" \
+ --notes "Release Notes: ${RELEASE_NOTES}"
Review Comment:
Could you download artifacts for RC and upload them?
Could you also ensure adding a new line at the end of file? (`utils-create-release-tarball.sh` also doesn't have the last new line.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2137697423
> However, I'm wondering if it makes sense to start that work now so that we can merge this PR in time for the next release.
Sounds like a great idea! I looked for a reference issue in JIRA that documents the process: https://issues.apache.org/jira/browse/INFRA-25610
Hm I just noticed that "All artifacts being signed can be built [reproducibly](https://reproducible-builds.org/)" is a requirement for automated signing. I don't know that we fulfill that yet (leaning towards no?). I'll have to look into it and see how hard this would be to implement. Though afaik we only sign the tarball, so that shouldn't be too hard to make work? Getting it to work with all binary artifacts would probably be a different story.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2150971336
I re-read the section again now and noticed that its section title is "Pushing commits to repositories".
This PR doesn't push any commits to apache/arrow. So the section may be unrelated to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1605051520
##########
.github/workflows/release_candidate.yml:
##########
@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ #"apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
Review Comment:
Yes, because we call `gh release create` in the last step of the workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1605050239
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
Review Comment:
In this case `GH_TOKEN` is necessary because we use GitHub CLI in the workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1607205784
##########
docs/source/developers/release.rst:
##########
@@ -200,6 +200,8 @@ Build source and binaries and submit them
.. code-block::
# Build the source release tarball and create Pull Request with verification tasks
+ #
+ # Note that you need to have GitHub CLI installed to run this script.
Review Comment:
Thanks for the suggestion @kou! I added a new script called `dev/release/utils-watch-gh-workflow.sh`. I modified `02-source.sh` to call this new script so that it only attempts to download the tarball from the GitHub Release once the `release_candidate.yml` workflow is finished. Additionally, `post-05-update-gh-release-notes.sh` calls `dev/release/utils-watch-gh-workflow.sh` to wait for the `release.yml` workflow to finish before attempting to update the release notes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2137742398
Thanks for looking into this @assignUser! I appreciate all the help!
In that case, I'll hold off on working on that step. Maybe we can do that in a separate PR once we can implement "reproducible build" requirement?
Thanks again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2053823172
> before we merge this
We also have to wait for infra to review this.
> I want to move the process that generates apache-arrow-X.Y.Z.tar.gz from 02-source.sh to .github/workflows/release_candidate.yml as a separated task.
Then let's also make the workflow sign it and adhere to the requirements listed [here](https://infra.apache.org/release-signing.html#automated-release-signing), which will also need a review by INFRA/security so we can get a 2 for 1 :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2050381210
Going to test out the new `release.yml` workflow on `mathworks/arrow`. I'll post a comment once I'm done with that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2052696552
I have opened a ticket with INFRA https://issues.apache.org/jira/browse/INFRA-25708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561894369
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
Review Comment:
The question is if we want to limit this to only respond to tags on these branches, which for the moment would be correct. Though it is rather unlikely that someone pushes a tag that matches the release tag to main or some other branch by accident... so :shrug: Better save than sorry?
Btw I don't remember where the `release-*` branches come from? Are they created after the vote? What is their purpose compared to `maint-*`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1604147006
##########
.github/workflows/dev.yml:
##########
@@ -124,6 +124,7 @@ jobs:
- name: Run Release Test
env:
ARROW_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Review Comment:
why is this `GH_TOKEN` necessary now?
##########
.github/workflows/release_candidate.yml:
##########
@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ #"apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
Review Comment:
again, is this necessary?
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
Review Comment:
As far as I understand cloning arrow is not necessary on this workflow as we are just downloading the release from the artifacts created for the RC and uploading the final release.
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
Review Comment:
is this necessary?
##########
docs/source/developers/release.rst:
##########
@@ -200,6 +200,8 @@ Build source and binaries and submit them
.. code-block::
# Build the source release tarball and create Pull Request with verification tasks
+ #
+ # Note that you need to have GitHub CLI installed to run this script.
Review Comment:
Should we add a note here saying that we have to wait for the workflow to finish creating the Release on GH before executing this? I usually execute those two scripts one after the other and if there is a minor queue on the runners and the release is not created this execution will fail to download the `tar.gz`. Adding a note to the release manager to validate the release is created could help prevent that.
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
the release notes are uploaded on the 4th step of the post release process instead and the tag is created on the first step. I don't think there's a problem with this but this link won't exist until the the website is updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561193073
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
I think we want to add both so it only triggers on the right tag on the right branch ('maint-*') to avoid issues with e.g. the cran release branch and tags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2050805038
I just noticed we might need INFRA approval before merginging this:
>Automated services MUST NOT push data to a repository or branch that is subject to official release as a software package by the project, unless the project secures specific prior authorization of the workflow from Infrastructure.
https://infra.apache.org/github-actions-policy.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561183492
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
Review Comment:
Oh I forgot to update the pattern before making the PR. Sorry about that. I'll update the pattern asap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2062909455
Thanks! Let's wait for reviews from others.
(We may need to ping them later.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1567966136
##########
matlab/tools/packageMatlabInterface.m:
##########
@@ -29,15 +29,10 @@
disp("Output Folder: " + outputFolder);
disp("Toolbox Version Raw: " + toolboxVersionRaw);
-
-% Note: This string processing heuristic may not be robust to future
-% changes in the Arrow versioning scheme.
-dotIdx = strfind(toolboxVersionRaw, ".");
-numDots = numel(dotIdx);
-if numDots >= 3
- toolboxVersion = extractBefore(toolboxVersionRaw, dotIdx(3));
-else
- toolboxVersion = toolboxVersionRaw;
+versionPattern = regexpPattern("[0-9]+.[0-9]+.[0-9]+");
Review Comment:
```suggestion
versionPattern = regexpPattern("^[0-9]+.[0-9]+.[0-9]+");
```
##########
dev/release/02-source.sh:
##########
@@ -25,6 +25,7 @@ set -eu
: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
: ${SOURCE_PR:=${SOURCE_DEFAULT}}
: ${SOURCE_VOTE:=${SOURCE_DEFAULT}}
+: ${SOURCE_DOWNLOAD:=${SOURCE_DEFAULT}}
Review Comment:
Could you move this to between `SOURCE_DEFAULT` and `SOURCE_RAT`?
We use the same order as the real process order for these `SOURCE_*`.
```shell
: ${SOURCE_DEFAULT:=1}
: ${SOURCE_DOWNLOAD:=${SOURCE_DEFAULT}}
: ${SOURCE_RAT:=${SOURCE_DEFAULT}}
...
if [ ${SOURCE_DOWNLOAD} -gt 0 ]; then
...
fi
if [ ${SOURCE_RAT} -gt 0 ]; then
...
fi
...
```
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,79 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
+ echo "Release Notes URL: ${release_notes}"
+ echo "RELEASE_NOTES=${release_notes}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --verify-tag \
+ --title "${RELEASE_TITLE}" \
+ --notes "Release Notes: ${RELEASE_NOTES}" \
+ release_candidate_artifacts/*
+
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2057150332
Sorry I accidentally closed the pull request - just re-opened it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1606970245
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
Should we now remove the release notes update here with the bad link?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2137728323
After looking into it we do sign a number of additional things (e.g. java, python and C# binaries...), so the reproducible build requirement might block this for a bit, sorry. I will look into what is actually required and how we can implement it.
Appreciate all your work on this @sgilmore10 !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561452507
##########
dev/release/post-16-delete-release-candidates.sh:
##########
Review Comment:
Deleted `post-16-delete-release-candidates.sh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2057172556
> +1 but we should wait for a review from @raulcd before we merge this. (@raulcd may review this after 16.0.0 release.)
>
> I want to move the process that generates `apache-arrow-X.Y.Z.tar.gz` from `02-source.sh` to `.github/workflows/release_candidate.yml` as a separated task. Something like:
>
> ```diff
> diff --git a/dev/release/02-source.sh b/dev/release/02-source.sh
> index 27ca67a76e..874c1955f8 100755
> --- a/dev/release/02-source.sh
> +++ b/dev/release/02-source.sh
> @@ -57,33 +57,12 @@ echo "Using commit $release_hash"
>
> tarball=apache-arrow-${version}.tar.gz
>
> -rm -rf ${tag}
> -# be conservative and use the release hash, even though git produces the same
> -# archive (identical hashes) using the scm tag
> -(cd "${SOURCE_TOP_DIR}" && \
> - git archive ${release_hash} --prefix ${tag}/) | \
> - tar xf -
> -
> -# Resolve all hard and symbolic links.
> -# If we change this, we must change ArrowSources.archive in
> -# dev/archery/archery/utils/source.py too.
> -rm -rf ${tag}.tmp
> -mv ${tag} ${tag}.tmp
> -cp -R -L ${tag}.tmp ${tag}
> -rm -rf ${tag}.tmp
> -
> -# Create a dummy .git/ directory to download the source files from GitHub with Source Link in C#.
> -dummy_git=${tag}/csharp/dummy.git
> -mkdir ${dummy_git}
> -pushd ${dummy_git}
> -echo ${release_hash} > HEAD
> -echo '[remote "origin"] url = https://github.com/apache/arrow.git' >> config
> -mkdir objects refs
> -popd
> -
> -# Create new tarball from modified source directory
> -tar czf ${tarball} ${tag}
> -rm -rf ${tag}
> +rm -f ${tarball}
> +gh release download \
> + ${tag} \
> + --repo apache/arrow \
> + --dir . \
> + --pattern "${tarball}"
>
> if [ ${SOURCE_RAT} -gt 0 ]; then
> "${SOURCE_DIR}/run-rat.sh" ${tarball}
> ```
This makes sense to me. However, I think it will require re-ordering the scripts to ensure we push the RC tag _**after**_ the crossbow job containing all build artifacts - specifically the MLTBX file - is finished. Otherwise, we won't be able to attach the MLTBX file - and potentially other build artifacts - as assets to the release candidate. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2059510714
Updates:
1. Added `dev/release/08-matlab-upload.sh` as @kou suggested. This script (1) signs the MLTBX artifacts downloaded from the `crossbow` job and (2) uploads the artifacts to the release candidate GitHub Release's area.
2. I noticed a small bug in `matlab/tools/packageMatlabInterface.m`. Specifically, the logic for extracting the version string only works on strings formatted like `${major}.${minor}.${patch}.dev${num})`. I rewrote this logic to support extracting the version string from strings formatted like `${major}.${minor}.${patch}-rc${rc_num}` as well.
3. Modified the name of the MLTBX file generated by `matlab/tools/packageMatlabInterface.m` to not include the release candidate number. Instead, the MLTBX file name just includes the version - this is similar to how the name of the tarball created by `dev/release/utils-create-release-tarball.sh` does not include the release candidate number.
Next Steps:
1. I still need to add the logic for signing the release tarball to the `release_candidate.yml` workflow. Once we hear back from INFRA regarding https://issues.apache.org/jira/browse/INFRA-25708, I'll submit a followup ticket requesting a signing key.
2. Given that I added `dev/release/08-matlab-upload.sh` to this PR, I'll close #40956 as it is now superfluous.
Let me know if you have any questions or comments!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2150049477
I think still we need INFRA's approval to use the GitHub Releases area based on the section @assignUser posted a month and half ago.
> Automated services MUST NOT push data to a repository or branch that is subject to official release as a software package by the project, unless the project secures specific prior authorization of the workflow from Infrastructure.
> https://infra.apache.org/github-actions-policy.html
I think we still need to wait? Regardless, I'll update `dev/release/02-source.sh` to sign the source tarball manually - unless I'm misunderstanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2167862516
After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 6ec2f222ff7a6ad9dcb3b5b7edc38c414259aee2.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/26230353804) has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561280468
##########
dev/release/02-source.sh:
##########
@@ -37,11 +37,10 @@ fi
version=$1
rc=$2
-tag=apache-arrow-${version}
+tag=apache-arrow-${version}-rc${rc}
Review Comment:
Ah ok. I'll update the `tarball` name to not include `-rcN`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2050035581
> I had a first look at the workflows only. I think we can just merge them into one and run|skip some steps with `if:` when the triggering tag has the `-rc` suffix.
That's a good point. I'll go ahead and merge the yml files together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2062154256
> I haven't tried this but I think that it will work.
Hi @Kou, I just tested out the most recent changes - with a few minor tweaks - using the mathworks/arrow repo. Here's a [link](https://github.com/mathworks/arrow/releases) to mathworks/arrow's GitHub Releases area. I'll keep these releases around for a day or so just in case anyone wants to take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1605059111
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
Review Comment:
Oh, that is a good point. I'll remove that step from the workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1606155062
##########
.github/workflows/release_candidate.yml:
##########
@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ #"apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
Review Comment:
```suggestion
# "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1606853700
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
Sounds good to me!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1607157893
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,78 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
+ name: Publish
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ fetch-depth: 0
+ - name: Get Tag Name of Latest Release Candidate
+ run: |
+ rc_tag=$(gh release list | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "RELEASE_CANDIDATE_TAG_NAME=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Store Version and Release Candidate Number
+ run: |
+ version_with_rc=${RELEASE_CANDIDATE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "VERSION_WITH_RC=${version_with_rc}" >> ${GITHUB_ENV}
+ echo "VERSION=${version}" >> ${GITHUB_ENV}
+ echo "RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Download Release Candidate Artifacts
+ run: |
+ mkdir release_candidate_artifacts
+ gh release download ${RELEASE_CANDIDATE_TAG_NAME} --dir release_candidate_artifacts
+ - name: Create Release Title
+ run: |
+ title="Apache Arrow ${VERSION}"
+ echo "RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes
+ run: |
+ release_notes="https://arrow.apache.org/release/${VERSION}.html"
Review Comment:
oh yes! Just pushed that change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561940410
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
Review Comment:
I don't think we want to trigger these workflows for branches. However, I could add a conditional to prevent the `Publish` job from running if the branch name doesn't match `release-*` or `maint-*`. How does that sound?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561896074
##########
.github/workflows/release.yml:
##########
@@ -0,0 +1,107 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+
+on:
+ push:
+ branches:
+ - "maint-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+"
+ - "release-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+ tags:
+ # Trigger workflow when a tag whose name matches either
+ # pattern "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" or pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+"
+ - "apache-arrow-[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish:
Review Comment:
Having two jobs might be better after all, I thought we could join it in one job but the required steps are just slightly different enough that some degree of duplication makes it much easier to read (I can not read through this without forgetting which step applies when xD :see_no_evil: ).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561287141
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Get Latest Release Candidate's Git Tag
+ run: |
+ rc_tag=$(gh release list --repo ${{github.repository}} | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "ARROW_RELEASE_CANDIDATE_TAG=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title=$(gh release view ${ARROW_RELEASE_CANDIDATE_TAG} --repo ${{github.repository}} --json name --jq .name | \
+ sed -E -e 's/ RC[0-9]+$//')
+ echo "Release Title: ${title}"
+ echo "ARROW_RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes URL
+ run: |
+ version=${GITHUB_REF_NAME#apache-arrow-}
+ release_note_url="https://arrow.apache.org/release/${version}.html"
+ echo "Release Notes URL: ${release_note_url}"
+ echo "ARROW_RELEASE_NOTES_URL=${release_note_url}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --title "${ARROW_RELEASE_TITLE}" \
+ --verify-tag \
+ --notes "Release Notes: ${ARROW_RELEASE_NOTES_URL}" \
+ --repo ${{github.repository}}
Review Comment:
I originally did include a step to download the artifacts from RC candidate. However, [gh release download errors](https://github.com/mathworks/arrow/actions/runs/8621826934/job/23631558469#step:3:32) if there aren't any assets to download. I planned on adding this step in #40956 since there'll be at least one asset to download at that point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2148735581
> I was under the impression INFRA would not give out a signing key to use only for signing the source archive, but I may be mistaken.
That's also my understanding (nothing would prevent us to sign non-reproducible binaries with the 'just for source'-key).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2148742574
> I think we still need to wait on INFRA to approve [INFRA-25708](https://issues.apache.org/jira/browse/INFRA-25708) before we can merge this PR.
I don't think that we need an approval from INFRA for this PR because this PR doesn't use automated signing. We sill sign manually in `dev/release/07-matlab-upload.sh`. It's same as the current our release process for our artifacts including the source archive, `.deb`/`.rpm` and so on.
> To clarify, do you think we could ask INFRA for a signing key for _just_ the source tarballs?
I think so.
I think that we can expand automated signing targets step by step. For example, we enable automated signing only for the source archive and then enable it for `.deb` too and so on.
> I was under the impression INFRA would not give out a signing key to use only for signing the source archive, but I may be mistaken.
Oh, I hadn't thought of that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2148767308
I pinged INFRA on slack, if they don't have time to review the workflow currently @kou proposed that we move the uploading and signing to the usual release scripts and merge the PR without automation. I agree with that approach, it will allow us to move this forward so matlab can be properly released with the upcoming 17.0.0 :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2118215191
Should be fixed now. Sorry about that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1606169901
##########
docs/source/developers/release.rst:
##########
@@ -200,6 +200,8 @@ Build source and binaries and submit them
.. code-block::
# Build the source release tarball and create Pull Request with verification tasks
+ #
+ # Note that you need to have GitHub CLI installed to run this script.
Review Comment:
FYI: We can wait for the workflow in our script like ADBC does: https://github.com/apache/arrow-adbc/blob/ddb1bcf7dfb249dc3e958678bf4d65bd205ca58f/dev/release/02-sign.sh#L37-L51
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2118247109
Just verified the updated versions of `release_candidate.yml` and `release.yml` work as expected as expected on `mathworks/arrow`.
1. Link to the `release_candidate.yml` workflow [run](https://github.com/mathworks/arrow/actions/runs/913324935).
1. Link to the `release.yml` workflow [run](https://github.com/mathworks/arrow/actions/runs/9133254955).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2152661335
The MATLAB check crashed during the `Run Tests` step on Windows after I rebased on main to deal with merge conflicts. I'm fairly certain that this crash is unrelated to the changes in this PR because the same crash occurred in the mathworks/arrow fork after syncing the branch to be up to date with apache/arrow: https://github.com/mathworks/arrow/actions/runs/9401909013/job/25894734169
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 closed pull request #41131: GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM})
URL: https://github.com/apache/arrow/pull/41131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1566296256
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
Based on the discussion, it seems like we should not trigger the release and release candidate workflows when pushing specific branches. Marking as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2058125281
Just an update:
After moving the release tarball generation to `release_candidate.yml` and using `gh release download` to download the tarball in `02-source.sh`, the source release check (`02-source-test.sh`) failed because the GitHub Release from which `02-source.sh` was trying to download the tarball did not exist.
To workaround this issue, I moved the tarball generation logic into a separate file called `dev/release/util-create-release-tarball.sh`, which `release_candidate.yml` calls to generate the release tarball. In `02-source.sh`, if an environment variable `TEST_RELEASE_SCRIPT` is defined and greater than 0, then `02-source.sh` invokes `util-create-release-tarball.sh` to generate the release tarball. Otherwise, it uses `gh release download` to retrieve the pre-built archive from the GitHub Releases area. Finally, I modified `02-source-test.rb` to ensure `TEST_RELEASE_SCRIPT` is defined and set to `1` when this script invokes `02-source.sh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561451498
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Store Release Tag Name as Environment Variable
+ # For convenience, store the release tag name as an environment
+ # variable for use in subsequent steps.
+ run: |
+ echo "ARROW_RELEASE_TAG_NAME=${{github.ref_name}}" >> "$GITHUB_ENV"
Review Comment:
Merged `publish_release.yml` and `publish_release_candidate.yml` into one file named `release.yml` and replaced all usage of `${{github.ref_name}}` with `${GITHUB_REF_NAME}` in `release.yml`. Marking as resolved.
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Store Release Tag Name as Environment Variable
+ # For convenience, store the release tag name as an environment
+ # variable for use in subsequent steps.
+ run: |
+ echo "ARROW_RELEASE_TAG_NAME=${{github.ref_name}}" >> "$GITHUB_ENV"
+ - name: Store Version and Release Candidate Number as Environment Variables
+ # From environment variable ARROW_RELEASE_TAG_NAME, extract the version and
+ # release candidate number. Store these values as environment variables.
+ run: |
+ version_with_rc=${ARROW_RELEASE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "ARROW_VERSION_WITH_RC=${version_with_rc}" >> "${GITHUB_ENV}"
+ echo "ARROW_VERSION=${version}" >> ${GITHUB_ENV}
+ echo "ARROW_RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release for Release Candidate
+ # Create a pre-release in apache/arrow's GitHub Releases area.
+ run: |
+ title="Apache Arrow ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ repository=${{github.repository}}
+ release_notes="Release Candidate: ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ gh release create \
+ ${ARROW_RELEASE_TAG_NAME} \
+ --prerelease \
+ --notes "${release_notes}" \
+ --title "${title}" \
+ --repo ${repository} \
+ --verify-tag
+ env:
+ GH_TOKEN: ${{ github.token }}
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1606783027
##########
docs/source/developers/release.rst:
##########
@@ -200,6 +200,8 @@ Build source and binaries and submit them
.. code-block::
# Build the source release tarball and create Pull Request with verification tasks
+ #
+ # Note that you need to have GitHub CLI installed to run this script.
Review Comment:
This is cool :) :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2148420389
> Given that the reproducible builds requirement may block adding automated release signing for a bit, how do you feel about adding that functionality in a separate PR?
+1
> If we go that route, I think I need to update `dev/release/02-source.sh` to sign the downloaded tarball and then upload the signed tarball to the GitHub Releases area.
We don't need to do it.
We don't need to work on automated release signing for both of the source archive and the MATLAB binary at once.
@raulcd Do you have any more comments? Can we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "sgilmore10 (via GitHub)" <gi...@apache.org>.
sgilmore10 commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2148220262
Hi @assignUser,
Given that the reproducible builds requirement may block adding automated release signing for a bit, how do you feel about adding that functionality in a separate PR? If we go that route, I think I need to update `dev/release/02-source.sh` to sign the downloaded tarball and then upload the signed tarball to the GitHub Releases area.
How does that sound?
Best,
Sarah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #41131:
URL: https://github.com/apache/arrow/pull/41131#issuecomment-2052689230
Nice, great progress!
>it made more sense to first add "generic" release candidate and release workflows
Oh yeah that makes sense, though I do think we should also add the official tarball + signature to the releases. Looking at the release scripts the only change to make would be to upload the tarball to svn before pushing the (rc) tag, which triggers this workflow where we could then pull from dist.a.o .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1560110161
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
Review Comment:
Please specify a more precise pattern here as we do use other tags (e.g. the go one)
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
Limit to release branches `maint-*` @kou or what do you think? I see we also have `release-*` branches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1560321890
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
We can remove this because we want to run this only for tags.
```suggestion
```
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushbranchestagsbranches-ignoretags-ignore
> If you define only tags/tags-ignore or only branches/branches-ignore, the workflow won't run for events affecting the undefined Git ref.
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Get Latest Release Candidate's Git Tag
+ run: |
+ rc_tag=$(gh release list --repo ${{github.repository}} | \
Review Comment:
```suggestion
rc_tag=$(gh release list | \
```
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
Review Comment:
```suggestion
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
Review Comment:
```suggestion
name: Release
```
##########
docs/source/developers/release.rst:
##########
@@ -189,8 +189,8 @@ Create the Release Candidate branch from the updated maintenance branch
# so for the first RC this would be: dev/release/01-prepare.sh 4.0.0 5.0.0 0
dev/release/01-prepare.sh <version> <next-version> <rc-number>
- # Push the release tag (for RC1 or later the --force flag is required)
- git push -u apache apache-arrow-<version>
+ # Push the release candidate tag (for RC1 or later the --force flag is required)
+ git push -u apache apache-arrow-<version>rc<rc-number>
Review Comment:
```suggestion
git push -u apache apache-arrow-<version>-rc<rc-number>
```
##########
.github/workflows/publish_release.yml:
##########
Review Comment:
How about renaming this to `release.yml`?
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
Review Comment:
```suggestion
# Trigger workflow when a tag whose name matches the pattern
# "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
Review Comment:
This is redundant.
```suggestion
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
Review Comment:
```suggestion
- "apache-arrow-*"
- "!apache-arrow-*-rc*"
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
Review Comment:
```suggestion
publish:
name: Publish
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
Review Comment:
If we remove this, we can omit `--repo` from `gh ...` because `gh` guess the default repository from the current Git repository:
```suggestion
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Get Latest Release Candidate's Git Tag
+ run: |
+ rc_tag=$(gh release list --repo ${{github.repository}} | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "ARROW_RELEASE_CANDIDATE_TAG=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title=$(gh release view ${ARROW_RELEASE_CANDIDATE_TAG} --repo ${{github.repository}} --json name --jq .name | \
Review Comment:
```suggestion
title=$(gh release view ${ARROW_RELEASE_CANDIDATE_TAG} --json name --jq .name | \
```
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Get Latest Release Candidate's Git Tag
+ run: |
+ rc_tag=$(gh release list --repo ${{github.repository}} | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "ARROW_RELEASE_CANDIDATE_TAG=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title=$(gh release view ${ARROW_RELEASE_CANDIDATE_TAG} --repo ${{github.repository}} --json name --jq .name | \
+ sed -E -e 's/ RC[0-9]+$//')
+ echo "Release Title: ${title}"
+ echo "ARROW_RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes URL
+ run: |
+ version=${GITHUB_REF_NAME#apache-arrow-}
+ release_note_url="https://arrow.apache.org/release/${version}.html"
+ echo "Release Notes URL: ${release_note_url}"
+ echo "ARROW_RELEASE_NOTES_URL=${release_note_url}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --title "${ARROW_RELEASE_TITLE}" \
+ --verify-tag \
+ --notes "Release Notes: ${ARROW_RELEASE_NOTES_URL}" \
+ --repo ${{github.repository}}
Review Comment:
```suggestion
--notes "Release Notes: ${ARROW_RELEASE_NOTES_URL}"
```
##########
.github/workflows/publish_release_candidate.yml:
##########
Review Comment:
How about renaming to `release_candidate.yml` or `rc.yml`?
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
+ tags:
+ # Trigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}" is pushed.
+ - "*"
+ - "!*-rc*"
+
+permissions:
+ contents: write
+
+env:
+ GH_TOKEN: ${{ github.token }}
+
+jobs:
+ publish-release:
+ name: Publish Release
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Get Latest Release Candidate's Git Tag
+ run: |
+ rc_tag=$(gh release list --repo ${{github.repository}} | \
+ cut -f3 | \
+ grep -F "${GITHUB_REF_NAME}-rc" | \
+ head -n1)
+ echo "Latest Release Candidate Tag: ${rc_tag}"
+ echo "ARROW_RELEASE_CANDIDATE_TAG=${rc_tag}" >> ${GITHUB_ENV}
+ - name: Create Release Title
+ run: |
+ title=$(gh release view ${ARROW_RELEASE_CANDIDATE_TAG} --repo ${{github.repository}} --json name --jq .name | \
+ sed -E -e 's/ RC[0-9]+$//')
+ echo "Release Title: ${title}"
+ echo "ARROW_RELEASE_TITLE=${title}" >> ${GITHUB_ENV}
+ - name: Create Release Notes URL
+ run: |
+ version=${GITHUB_REF_NAME#apache-arrow-}
+ release_note_url="https://arrow.apache.org/release/${version}.html"
+ echo "Release Notes URL: ${release_note_url}"
+ echo "ARROW_RELEASE_NOTES_URL=${release_note_url}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release
+ run: |
+ gh release create ${GITHUB_REF_NAME} \
+ --title "${ARROW_RELEASE_TITLE}" \
+ --verify-tag \
+ --notes "Release Notes: ${ARROW_RELEASE_NOTES_URL}" \
+ --repo ${{github.repository}}
Review Comment:
Could you also copy artifacts in the RC like https://github.com/apache/arrow-flight-sql-postgresql/blob/6e013be8491305295c76632b32f21b84296e5bb0/.github/workflows/release.yaml#L48-L53 ?
(Or we can do it when we work on #40956.)
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
Review Comment:
```suggestion
name: Release Candidate
```
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
Review Comment:
```suggestion
# Trigger workflow when a tag whose name matches the pattern
# "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
```
##########
dev/release/02-source.sh:
##########
@@ -37,11 +37,10 @@ fi
version=$1
rc=$2
-tag=apache-arrow-${version}
+tag=apache-arrow-${version}-rc${rc}
Review Comment:
This change is OK but we want to use `apache-arrow-X.Y.Z.tar.gz` not `apache-arrow-X.Y.Z-rcN.tar.gz` for archive name.
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Store Release Tag Name as Environment Variable
+ # For convenience, store the release tag name as an environment
+ # variable for use in subsequent steps.
+ run: |
+ echo "ARROW_RELEASE_TAG_NAME=${{github.ref_name}}" >> "$GITHUB_ENV"
Review Comment:
We can refer `github.ref_name` as `${GITHUB_REF_NAME}`.
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Store Release Tag Name as Environment Variable
+ # For convenience, store the release tag name as an environment
+ # variable for use in subsequent steps.
+ run: |
+ echo "ARROW_RELEASE_TAG_NAME=${{github.ref_name}}" >> "$GITHUB_ENV"
+ - name: Store Version and Release Candidate Number as Environment Variables
+ # From environment variable ARROW_RELEASE_TAG_NAME, extract the version and
+ # release candidate number. Store these values as environment variables.
+ run: |
+ version_with_rc=${ARROW_RELEASE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "ARROW_VERSION_WITH_RC=${version_with_rc}" >> "${GITHUB_ENV}"
+ echo "ARROW_VERSION=${version}" >> ${GITHUB_ENV}
+ echo "ARROW_RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release for Release Candidate
+ # Create a pre-release in apache/arrow's GitHub Releases area.
+ run: |
+ title="Apache Arrow ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ repository=${{github.repository}}
+ release_notes="Release Candidate: ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ gh release create \
+ ${ARROW_RELEASE_TAG_NAME} \
+ --prerelease \
+ --notes "${release_notes}" \
+ --title "${title}" \
+ --repo ${repository} \
Review Comment:
```suggestion
```
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Store Release Tag Name as Environment Variable
+ # For convenience, store the release tag name as an environment
+ # variable for use in subsequent steps.
+ run: |
+ echo "ARROW_RELEASE_TAG_NAME=${{github.ref_name}}" >> "$GITHUB_ENV"
+ - name: Store Version and Release Candidate Number as Environment Variables
+ # From environment variable ARROW_RELEASE_TAG_NAME, extract the version and
+ # release candidate number. Store these values as environment variables.
+ run: |
+ version_with_rc=${ARROW_RELEASE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "ARROW_VERSION_WITH_RC=${version_with_rc}" >> "${GITHUB_ENV}"
+ echo "ARROW_VERSION=${version}" >> ${GITHUB_ENV}
+ echo "ARROW_RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release for Release Candidate
+ # Create a pre-release in apache/arrow's GitHub Releases area.
+ run: |
+ title="Apache Arrow ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ repository=${{github.repository}}
+ release_notes="Release Candidate: ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ gh release create \
+ ${ARROW_RELEASE_TAG_NAME} \
+ --prerelease \
+ --notes "${release_notes}" \
+ --title "${title}" \
+ --repo ${repository} \
+ --verify-tag
+ env:
+ GH_TOKEN: ${{ github.token }}
Review Comment:
Could you add a new line at the end of file?
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
Review Comment:
```suggestion
publish:
name: Publish to GitHub Releases
```
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
Review Comment:
```suggestion
```
##########
dev/release/post-02-binary.sh:
##########
@@ -108,3 +108,14 @@ docker_run \
VERBOSE=${VERBOSE:-no} \
VERSION=${version} \
YUM_TARGETS=$(IFS=,; echo "${yum_targets[*]}")
+
+# Create the release tag and trigger the Publish Release workflow.
+release_candidate_tag=apache-arrow-${version}-rc${num}
+release_tag=apache-arrow-${version}
+git tag -a ${release_tag} ${release_candidate_tag}^{} -m "[Release] Apache Arrow Release ${version}"
+git push origin ${release_tag}
+
+# Wait for the Publish Release workflow to finish.
+sleep 2s
+workflow_id=$(gh run list --repo apache/arrow --workflow "Publish Release" | cut -f7 | head -n1)
+gh run watch ${workflow_id} --repo apache/arrow
Review Comment:
Could you do this in `dev/release/post-01-tag.sh`?
(We need to create `dev/release/post-01-tag.sh` and increment other `post-*.sh`.)
##########
docs/source/developers/release.rst:
##########
@@ -189,8 +189,8 @@ Create the Release Candidate branch from the updated maintenance branch
# so for the first RC this would be: dev/release/01-prepare.sh 4.0.0 5.0.0 0
dev/release/01-prepare.sh <version> <next-version> <rc-number>
- # Push the release tag (for RC1 or later the --force flag is required)
- git push -u apache apache-arrow-<version>
+ # Push the release candidate tag (for RC1 or later the --force flag is required)
Review Comment:
```suggestion
# Push the release candidate tag
```
##########
dev/release/post-16-delete-release-candidates.sh:
##########
Review Comment:
I think that we don't need this because there is no limit on the total size of a release: https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases#storage-and-bandwidth-quotas
##########
.github/workflows/publish_release_candidate.yml:
##########
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release Candidate
+
+on:
+ push:
+ tags:
+ # Rrigger workflow when a tag whose name matches the pattern
+ # "apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}" is pushed.
+ - "apache-arrow-*-rc*"
+
+jobs:
+ publish-release-candidate:
+ name: Publish Release Candidate to GitHub Releases
+ runs-on: ubuntu-latest
+ if: github.ref_type == 'tag'
+ steps:
+ - name: Checkout apache/arrow
+ uses: actions/checkout@v4
+ with:
+ path: arrow
+ fetch-depth: 0
+ - name: Store Release Tag Name as Environment Variable
+ # For convenience, store the release tag name as an environment
+ # variable for use in subsequent steps.
+ run: |
+ echo "ARROW_RELEASE_TAG_NAME=${{github.ref_name}}" >> "$GITHUB_ENV"
+ - name: Store Version and Release Candidate Number as Environment Variables
+ # From environment variable ARROW_RELEASE_TAG_NAME, extract the version and
+ # release candidate number. Store these values as environment variables.
+ run: |
+ version_with_rc=${ARROW_RELEASE_TAG_NAME#apache-arrow-}
+ version=${version_with_rc%-rc*}
+ rc_num=${version_with_rc#${version}-rc}
+ echo "ARROW_VERSION_WITH_RC=${version_with_rc}" >> "${GITHUB_ENV}"
+ echo "ARROW_VERSION=${version}" >> ${GITHUB_ENV}
+ echo "ARROW_RC_NUM=${rc_num}" >> ${GITHUB_ENV}
+ - name: Create GitHub Release for Release Candidate
+ # Create a pre-release in apache/arrow's GitHub Releases area.
+ run: |
+ title="Apache Arrow ${ARROW_VERSION} RC${ARROW_RC_NUM}"
+ repository=${{github.repository}}
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41102: [Packaging][Release] Create unique git tags for release candidates (e.g. apache-arrow-{MAJOR}.{MINOR}.{PATCH}-rc{RC_NUM}) [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on code in PR #41131:
URL: https://github.com/apache/arrow/pull/41131#discussion_r1561191436
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
No we
##########
.github/workflows/publish_release.yml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Publish Release
+
+on:
+ push:
+ branches:
+ - "!**"
Review Comment:
No we
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org