You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "adnanhemani (via GitHub)" <gi...@apache.org> on 2026/04/16 08:09:50 UTC
[PR] Add PyPI Publishing to Automated Release [polaris]
adnanhemani opened a new pull request, #4220:
URL: https://github.com/apache/polaris/pull/4220
Changes the PyPI nightly build script to use Trusted Publishers, as well as adding publishing scripts when generating a new release candidate and publishing new Polaris versions.
I've tested this from my fork: https://github.com/adnanhemani/polaris/actions/runs/24499401086
I've already set up Trusted Publishers for both PyPI and TestPyPI:
<img width="1918" height="1011" alt="Screenshot 2026-04-16 at 1 06 56โฏAM" src="https://github.com/user-attachments/assets/3e4fae07-b3f6-49f4-878b-18a8876316e5" />
<img width="1923" height="1010" alt="Screenshot 2026-04-16 at 1 07 18โฏAM" src="https://github.com/user-attachments/assets/b4d7fa25-cd4d-42f8-b755-7ac0efda8c96" />
## Checklist
- [X] ๐ก๏ธ Don't disclose security issues!
- [ ] ๐ Clearly explained why the changes are needed, or linked related issues: Fixes #
- [ ] ๐งช Added/updated tests with good coverage, or manually tested (and explained how)
- [ ] ๐ก Added comments for complex logic
- [ ] ๐งพ Updated `CHANGELOG.md` (if needed)
- [ ] ๐ Updated documentation in `site/content/in-dev/unreleased` (if needed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3094702871
##########
Makefile:
##########
@@ -188,18 +188,31 @@ client-lint: client-setup-env ## Run linting checks for Polaris client
@$(ACTIVATE_AND_CD) && uv run --active pre-commit run --files integration_tests/* generate_clients.py apache_polaris/cli/* apache_polaris/cli/command/* apache_polaris/cli/options/* test/*
@echo "--- Client linting checks complete ---"
-.PHONY: client-nightly-publish
-client-nightly-publish: client-setup-env ## Build and publish nightly version to Test PyPI
- @echo "--- Starting nightly publish ---"
+# Internal target: stamp version and build. Requires BUILD_VERSION to be set.
+.PHONY: _client-build
+_client-build: client-setup-env
@$(ACTIVATE_AND_CD) && \
- CURRENT_VERSION=$$(uv version --short) && \
- DATE_SUFFIX=$$(date -u +%Y%m%d%H%M%S) && \
- NIGHTLY_VERSION="$${CURRENT_VERSION}.dev$${DATE_SUFFIX}" && \
- echo "Publishing nightly version: $${NIGHTLY_VERSION}" && \
- uv version "$${NIGHTLY_VERSION}" && \
- uv build --clear && \
- uv publish --index testpypi
- @echo "--- Nightly publish complete ---"
+ echo "Building version: $(BUILD_VERSION)" && \
+ uv version "$(BUILD_VERSION)" && \
+ uv build --clear
Review Comment:
Instead of making an internal one, could we re-use client-build and extend it to optionally set the version?
Currently, CLI's License & Notice are only valid if we only publish the wheel (it will be a universal wheel): https://github.com/apache/polaris/pull/3891#discussion_r2881190870. So we probably need `FORMAT=wheel` when building artifacts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3104043005
##########
.github/workflows/nightly.yml:
##########
@@ -93,8 +96,13 @@ jobs:
distribution: 'temurin'
java-version: '21'
- - name: Publish Python client to Test PyPI
- env:
- UV_PUBLISH_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }} # zizmor: ignore[secrets-outside-env]
+ - name: Build Python client nightly
run: |
- make client-nightly-publish
+ make client-nightly-build
Review Comment:
Resolved as per comment below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3096702509
##########
.github/workflows/nightly.yml:
##########
@@ -76,6 +76,8 @@ jobs:
nightly_build_python_client:
runs-on: ubuntu-latest
if: github.repository == 'apache/polaris'
+ permissions:
+ id-token: write
Review Comment:
Done!
##########
.github/workflows/nightly.yml:
##########
@@ -76,6 +76,8 @@ jobs:
nightly_build_python_client:
runs-on: ubuntu-latest
if: github.repository == 'apache/polaris'
+ permissions:
+ id-token: write
Review Comment:
Done!
##########
.github/workflows/release-3-build-and-publish-artifacts.yml:
##########
@@ -523,10 +523,62 @@ jobs:
| Artifact Hub metadata | โ
Committed to dist dev |
EOT
+ publish-python-client-rc:
+ name: Publish Python Client RC to PyPI
+ runs-on: ubuntu-latest
+ needs: [prerequisite-checks]
+ permissions:
+ contents: read
+ id-token: write
+ env:
+ DRY_RUN: ${{ needs.prerequisite-checks.outputs.dry_run }}
+ version_without_rc: ${{ needs.prerequisite-checks.outputs.version_without_rc }}
+ rc_number: ${{ needs.prerequisite-checks.outputs.rc_number }}
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+
+ - name: Set up Python
+ uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
+ with:
+ python-version: "3.13"
+
+ - name: Set up JDK for openapi-generator-cli
+ uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
+ with:
+ distribution: 'temurin'
+ java-version: '21'
+
+ - name: Build Python client RC
+ run: |
+ make client-rc-build RC_VERSION="${version_without_rc}" RC_NUMBER="${rc_number}"
+
+ - name: Publish Python client RC to Test PyPI
+ uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
+ with:
+ repository-url: https://test.pypi.org/legacy/
+ packages-dir: client/python/dist/
+
+ - name: Update step summary
+ run: |
+ echo "## Python Client RC" >> $GITHUB_STEP_SUMMARY
+ cat <<EOT >> $GITHUB_STEP_SUMMARY
+ ๐ Python client RC published successfully:
+
+ | Property | Value |
+ | --- | --- |
+ | Version | \`${version_without_rc}rc${rc_number}\` |
+ | Registry | Test PyPI |
+ EOT
+
generate-release-email:
name: Generate Release Email Body
runs-on: ubuntu-latest
- needs: [prerequisite-checks, build-and-publish-artifacts, build-docker, build-and-stage-helm-chart]
+ needs: [prerequisite-checks, build-and-publish-artifacts, build-docker, build-and-stage-helm-chart, publish-python-client-rc]
Review Comment:
Done.
##########
.github/workflows/release-3-build-and-publish-artifacts.yml:
##########
@@ -523,10 +523,62 @@ jobs:
| Artifact Hub metadata | โ
Committed to dist dev |
EOT
+ publish-python-client-rc:
+ name: Publish Python Client RC to PyPI
+ runs-on: ubuntu-latest
+ needs: [prerequisite-checks]
+ permissions:
+ contents: read
+ id-token: write
Review Comment:
Done.
##########
Makefile:
##########
@@ -188,18 +188,31 @@ client-lint: client-setup-env ## Run linting checks for Polaris client
@$(ACTIVATE_AND_CD) && uv run --active pre-commit run --files integration_tests/* generate_clients.py apache_polaris/cli/* apache_polaris/cli/command/* apache_polaris/cli/options/* test/*
@echo "--- Client linting checks complete ---"
-.PHONY: client-nightly-publish
-client-nightly-publish: client-setup-env ## Build and publish nightly version to Test PyPI
- @echo "--- Starting nightly publish ---"
+# Internal target: stamp version and build. Requires BUILD_VERSION to be set.
+.PHONY: _client-build
+_client-build: client-setup-env
@$(ACTIVATE_AND_CD) && \
- CURRENT_VERSION=$$(uv version --short) && \
- DATE_SUFFIX=$$(date -u +%Y%m%d%H%M%S) && \
- NIGHTLY_VERSION="$${CURRENT_VERSION}.dev$${DATE_SUFFIX}" && \
- echo "Publishing nightly version: $${NIGHTLY_VERSION}" && \
- uv version "$${NIGHTLY_VERSION}" && \
- uv build --clear && \
- uv publish --index testpypi
- @echo "--- Nightly publish complete ---"
+ echo "Building version: $(BUILD_VERSION)" && \
+ uv version "$(BUILD_VERSION)" && \
+ uv build --clear
Review Comment:
Good call, changed.
##########
.github/workflows/release-4-publish-release.yml:
##########
@@ -38,6 +38,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
+ id-token: write
Review Comment:
Moved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani commented on PR #4220:
URL: https://github.com/apache/polaris/pull/4220#issuecomment-4291524706
@snazy I agree with you - that's why I separated the PyPI uploads into separate Jobs. Did I miss something there?
I also don't disagree with your comments that we should separate all these steps into different jobs - but I don't find that necessary in _this_ PR. I would highly prefer to do that in a follow up PR to reduce the scope of this PR. Is this a blocker for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3102112902
##########
.github/workflows/nightly.yml:
##########
@@ -93,8 +96,13 @@ jobs:
distribution: 'temurin'
java-version: '21'
- - name: Publish Python client to Test PyPI
- env:
- UV_PUBLISH_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }} # zizmor: ignore[secrets-outside-env]
+ - name: Build Python client nightly
run: |
- make client-nightly-publish
+ make client-nightly-build
Review Comment:
i tried running this locally, got an error
```
uv and dependencies installed.
--- Building client distribution ---
error: after parsing `1.4.0.dev20260417172537`, found `.dev20260417172701`, which is not part of a valid version
make[1]: *** [client-build] Error 2
make: *** [client-nightly-build] Error 2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani commented on PR #4220:
URL: https://github.com/apache/polaris/pull/4220#issuecomment-4325080092
Leaving this PR open for 24hr more for any final feedback before I merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani commented on PR #4220:
URL: https://github.com/apache/polaris/pull/4220#issuecomment-4263962163
@kevinjqliu - we are pushing RCs to TestPyPI and only pushing the released versions to PyPI. I believe this should be good enough to not cause errant versions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani merged PR #4220:
URL: https://github.com/apache/polaris/pull/4220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3093091625
##########
.github/workflows/nightly.yml:
##########
@@ -76,6 +76,8 @@ jobs:
nightly_build_python_client:
runs-on: ubuntu-latest
if: github.repository == 'apache/polaris'
+ permissions:
+ id-token: write
Review Comment:
This permission isn't commonly known, please add a comment explaining what it's used for.
##########
.github/workflows/release-4-publish-release.yml:
##########
@@ -38,6 +38,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
+ id-token: write
Review Comment:
This one concerns me, because _all_ steps now get access to OIDC/trusted-publishing.
##########
.github/workflows/release-3-build-and-publish-artifacts.yml:
##########
@@ -523,10 +523,62 @@ jobs:
| Artifact Hub metadata | โ
Committed to dist dev |
EOT
+ publish-python-client-rc:
+ name: Publish Python Client RC to PyPI
+ runs-on: ubuntu-latest
+ needs: [prerequisite-checks]
+ permissions:
+ contents: read
+ id-token: write
+ env:
+ DRY_RUN: ${{ needs.prerequisite-checks.outputs.dry_run }}
+ version_without_rc: ${{ needs.prerequisite-checks.outputs.version_without_rc }}
+ rc_number: ${{ needs.prerequisite-checks.outputs.rc_number }}
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+
+ - name: Set up Python
+ uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
+ with:
+ python-version: "3.13"
+
+ - name: Set up JDK for openapi-generator-cli
+ uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
+ with:
+ distribution: 'temurin'
+ java-version: '21'
+
+ - name: Build Python client RC
+ run: |
+ make client-rc-build RC_VERSION="${version_without_rc}" RC_NUMBER="${rc_number}"
+
+ - name: Publish Python client RC to Test PyPI
+ uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
+ with:
+ repository-url: https://test.pypi.org/legacy/
+ packages-dir: client/python/dist/
+
+ - name: Update step summary
+ run: |
+ echo "## Python Client RC" >> $GITHUB_STEP_SUMMARY
+ cat <<EOT >> $GITHUB_STEP_SUMMARY
+ ๐ Python client RC published successfully:
+
+ | Property | Value |
+ | --- | --- |
+ | Version | \`${version_without_rc}rc${rc_number}\` |
+ | Registry | Test PyPI |
+ EOT
+
generate-release-email:
name: Generate Release Email Body
runs-on: ubuntu-latest
- needs: [prerequisite-checks, build-and-publish-artifacts, build-docker, build-and-stage-helm-chart]
+ needs: [prerequisite-checks, build-and-publish-artifacts, build-docker, build-and-stage-helm-chart, publish-python-client-rc]
Review Comment:
Nit: the list is growing, maybe migrate to one line per dependency.
##########
.github/workflows/release-3-build-and-publish-artifacts.yml:
##########
@@ -523,10 +523,62 @@ jobs:
| Artifact Hub metadata | โ
Committed to dist dev |
EOT
+ publish-python-client-rc:
+ name: Publish Python Client RC to PyPI
+ runs-on: ubuntu-latest
+ needs: [prerequisite-checks]
+ permissions:
+ contents: read
+ id-token: write
Review Comment:
Same here, deserves a comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3094702871
##########
Makefile:
##########
@@ -188,18 +188,31 @@ client-lint: client-setup-env ## Run linting checks for Polaris client
@$(ACTIVATE_AND_CD) && uv run --active pre-commit run --files integration_tests/* generate_clients.py apache_polaris/cli/* apache_polaris/cli/command/* apache_polaris/cli/options/* test/*
@echo "--- Client linting checks complete ---"
-.PHONY: client-nightly-publish
-client-nightly-publish: client-setup-env ## Build and publish nightly version to Test PyPI
- @echo "--- Starting nightly publish ---"
+# Internal target: stamp version and build. Requires BUILD_VERSION to be set.
+.PHONY: _client-build
+_client-build: client-setup-env
@$(ACTIVATE_AND_CD) && \
- CURRENT_VERSION=$$(uv version --short) && \
- DATE_SUFFIX=$$(date -u +%Y%m%d%H%M%S) && \
- NIGHTLY_VERSION="$${CURRENT_VERSION}.dev$${DATE_SUFFIX}" && \
- echo "Publishing nightly version: $${NIGHTLY_VERSION}" && \
- uv version "$${NIGHTLY_VERSION}" && \
- uv build --clear && \
- uv publish --index testpypi
- @echo "--- Nightly publish complete ---"
+ echo "Building version: $(BUILD_VERSION)" && \
+ uv version "$(BUILD_VERSION)" && \
+ uv build --clear
Review Comment:
Instead of making an internal one, could we re-use client-build and extend it to optionally set the version?
Currently, CLI's License & Notice are only valid if we only publish the wheel (it will be a universal wheel): https://github.com/apache/polaris/pull/3891#discussion_r2881190870. So we probably need `FORMAT=wheel` when building artifacts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3095260231
##########
.github/workflows/nightly.yml:
##########
@@ -76,6 +76,8 @@ jobs:
nightly_build_python_client:
runs-on: ubuntu-latest
if: github.repository == 'apache/polaris'
+ permissions:
+ id-token: write
Review Comment:
```suggestion
permissions:
# IMPORTANT: this permission is mandatory for Trusted Publishing
id-token: write
```
https://docs.pypi.org/trusted-publishers/using-a-publisher/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.
adnanhemani commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3135593333
##########
Makefile:
##########
@@ -188,18 +191,22 @@ client-lint: client-setup-env ## Run linting checks for Polaris client
@$(ACTIVATE_AND_CD) && uv run --active pre-commit run --files integration_tests/* generate_clients.py apache_polaris/cli/* apache_polaris/cli/command/* apache_polaris/cli/options/* test/*
@echo "--- Client linting checks complete ---"
-.PHONY: client-nightly-publish
-client-nightly-publish: client-setup-env ## Build and publish nightly version to Test PyPI
- @echo "--- Starting nightly publish ---"
- @$(ACTIVATE_AND_CD) && \
- CURRENT_VERSION=$$(uv version --short) && \
- DATE_SUFFIX=$$(date -u +%Y%m%d%H%M%S) && \
- NIGHTLY_VERSION="$${CURRENT_VERSION}.dev$${DATE_SUFFIX}" && \
- echo "Publishing nightly version: $${NIGHTLY_VERSION}" && \
- uv version "$${NIGHTLY_VERSION}" && \
- uv build --clear && \
- uv publish --index testpypi
- @echo "--- Nightly publish complete ---"
+.PHONY: client-nightly-build
+client-nightly-build: client-setup-env ## Build nightly version for publishing to Test PyPI
+ @$(MAKE) client-build \
+ VERSION="$$($(VENV_DIR)/bin/uv --directory $(PYTHON_CLIENT_DIR) version --short).dev$$(date -u +%Y%m%d%H%M%S)" \
+ FORMAT=wheel
+
+.PHONY: client-rc-build
+client-rc-build: ## Build RC version for publishing to Test PyPI (requires RC_VERSION and RC_NUMBER)
+ @if [ -z "$(RC_VERSION)" ]; then echo "ERROR: RC_VERSION is not set"; exit 1; fi
+ @if [ -z "$(RC_NUMBER)" ]; then echo "ERROR: RC_NUMBER is not set"; exit 1; fi
+ @$(MAKE) client-build VERSION="$(RC_VERSION)rc$(RC_NUMBER)"
Review Comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on PR #4220:
URL: https://github.com/apache/polaris/pull/4220#issuecomment-4288010287
The [recent Polaris 1.4.0 release workflow failed quite early](https://github.com/apache/polaris/actions/runs/24698069238/job/72238085638) during the changes to Apache SVN, before it handles Helm charts, Docker images or Nexus or even create the GitHub release. I think, Helm charts, Docker images and Nexus jobs would have worked fine during that workflow run, if those would have been separate _jobs_ in the workflow.
The unsolvable issue with release-related workflows is that those steps cannot be integration tested. This is why I think splitting the work across multiple jobs reduces the amount of non-executed steps, that require manual work later.
Adding PyPi publishing adds another step that can potentially fail a whole release, but having that as a separate workflow _job_ reduces the risk of affecting other parts of the release automation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3095267632
##########
.github/workflows/release-4-publish-release.yml:
##########
@@ -38,6 +38,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
+ id-token: write
Review Comment:
+1 we can move this to the step that pushes to pypi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3091671298
##########
.github/workflows/release-4-publish-release.yml:
##########
@@ -431,6 +432,34 @@
# Use the Gradle task to release the Apache staging repository with the specific staging repository ID
exec_process ./gradlew releaseApacheStagingRepository --staging-repository-id "${STAGING_REPOSITORY_ID}"
+ cat <<EOT >> $GITHUB_STEP_SUMMARY
+ ## Nexus
+ โ
Nexus staging repository released
+ EOT
+
+ - name: Set up Python
+ uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
+ with:
+ python-version: "3.13"
+
+ - name: Build Python client release
+ run: |
+ make client-release-build RELEASE_VERSION="${version_without_rc}"
+
+ - name: Publish Python client release to PyPI
+ uses: pypa/gh-action-pypi-publish@release/v1
Review Comment:
## zizmor /
unpinned action reference
[Show more details](https://github.com/apache/polaris/security/code-scanning/46)
##########
.github/workflows/release-3-build-and-publish-artifacts.yml:
##########
@@ -523,10 +523,62 @@
| Artifact Hub metadata | โ
Committed to dist dev |
EOT
+ publish-python-client-rc:
+ name: Publish Python Client RC to PyPI
+ runs-on: ubuntu-latest
+ needs: [prerequisite-checks]
+ permissions:
+ contents: read
+ id-token: write
+ env:
+ DRY_RUN: ${{ needs.prerequisite-checks.outputs.dry_run }}
+ version_without_rc: ${{ needs.prerequisite-checks.outputs.version_without_rc }}
+ rc_number: ${{ needs.prerequisite-checks.outputs.rc_number }}
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+
+ - name: Set up Python
+ uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
+ with:
+ python-version: "3.13"
+
+ - name: Set up JDK for openapi-generator-cli
+ uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
+ with:
+ distribution: 'temurin'
+ java-version: '21'
+
+ - name: Build Python client RC
+ run: |
+ make client-rc-build RC_VERSION="${version_without_rc}" RC_NUMBER="${rc_number}"
+
+ - name: Publish Python client RC to Test PyPI
+ uses: pypa/gh-action-pypi-publish@release/v1
Review Comment:
## zizmor /
unpinned action reference
[Show more details](https://github.com/apache/polaris/security/code-scanning/45)
##########
.github/workflows/nightly.yml:
##########
@@ -93,8 +94,13 @@
distribution: 'temurin'
java-version: '21'
- - name: Publish Python client to Test PyPI
- env:
- UV_PUBLISH_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }} # zizmor: ignore[secrets-outside-env]
+ - name: Build Python client nightly
run: |
- make client-nightly-publish
+ make client-nightly-build
+
+ - name: Publish Python client to Test PyPI
+ uses: pypa/gh-action-pypi-publish@release/v1
Review Comment:
## zizmor /
unpinned action reference
[Show more details](https://github.com/apache/polaris/security/code-scanning/44)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3127309753
##########
Makefile:
##########
@@ -188,18 +191,22 @@ client-lint: client-setup-env ## Run linting checks for Polaris client
@$(ACTIVATE_AND_CD) && uv run --active pre-commit run --files integration_tests/* generate_clients.py apache_polaris/cli/* apache_polaris/cli/command/* apache_polaris/cli/options/* test/*
@echo "--- Client linting checks complete ---"
-.PHONY: client-nightly-publish
-client-nightly-publish: client-setup-env ## Build and publish nightly version to Test PyPI
- @echo "--- Starting nightly publish ---"
- @$(ACTIVATE_AND_CD) && \
- CURRENT_VERSION=$$(uv version --short) && \
- DATE_SUFFIX=$$(date -u +%Y%m%d%H%M%S) && \
- NIGHTLY_VERSION="$${CURRENT_VERSION}.dev$${DATE_SUFFIX}" && \
- echo "Publishing nightly version: $${NIGHTLY_VERSION}" && \
- uv version "$${NIGHTLY_VERSION}" && \
- uv build --clear && \
- uv publish --index testpypi
- @echo "--- Nightly publish complete ---"
+.PHONY: client-nightly-build
+client-nightly-build: client-setup-env ## Build nightly version for publishing to Test PyPI
+ @$(MAKE) client-build \
+ VERSION="$$($(VENV_DIR)/bin/uv --directory $(PYTHON_CLIENT_DIR) version --short).dev$$(date -u +%Y%m%d%H%M%S)" \
+ FORMAT=wheel
+
+.PHONY: client-rc-build
+client-rc-build: ## Build RC version for publishing to Test PyPI (requires RC_VERSION and RC_NUMBER)
+ @if [ -z "$(RC_VERSION)" ]; then echo "ERROR: RC_VERSION is not set"; exit 1; fi
+ @if [ -z "$(RC_NUMBER)" ]; then echo "ERROR: RC_NUMBER is not set"; exit 1; fi
+ @$(MAKE) client-build VERSION="$(RC_VERSION)rc$(RC_NUMBER)"
Review Comment:
Per offline discussion on the LIcense issue, how about let's put `FORMAT=wheel` in both rc-build and release-build for now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add PyPI Publishing to Automated Release [polaris]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #4220:
URL: https://github.com/apache/polaris/pull/4220#discussion_r3102300533
##########
Makefile:
##########
@@ -127,18 +127,21 @@ client-install-dependencies: $(VENV_DIR)
client-setup-env: $(VENV_DIR) client-install-dependencies
.PHONY: client-build
-client-build: client-setup-env ## Build client distribution. Pass FORMAT=sdist or FORMAT=wheel to build a specific format.
+client-build: client-setup-env ## Build client distribution. Pass FORMAT=sdist or FORMAT=wheel to build a specific format, and VERSION to stamp the version before building.
@echo "--- Building client distribution ---"
+ @if [ -n "$(VERSION)" ]; then \
+ $(ACTIVATE_AND_CD) && uv version "$(VERSION)"; \
+ fi
@if [ -n "$(FORMAT)" ]; then \
if [ "$(FORMAT)" != "sdist" ] && [ "$(FORMAT)" != "wheel" ]; then \
echo "Error: Invalid format '$(FORMAT)'. Supported formats are 'sdist' and 'wheel'." >&2; \
exit 1; \
fi; \
echo "Building with format: $(FORMAT)"; \
- $(ACTIVATE_AND_CD) && uv build --format $(FORMAT); \
+ $(ACTIVATE_AND_CD) && uv build --clear --format $(FORMAT); \
Review Comment:
```suggestion
$(ACTIVATE_AND_CD) && uv build --clear --$(FORMAT); \
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org