You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "kumaab (via GitHub)" <gi...@apache.org> on 2025/07/18 17:16:13 UTC
[PR] RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry [ranger]
kumaab opened a new pull request, #613:
URL: https://github.com/apache/ranger/pull/613
## What changes were proposed in this pull request?
A new workflow to build ranger images: `ranger, ranger-db, ranger-solr and ranger-zk` with multi-arch support and publish them to GitHub Container Registry and optionally to DockerHub.
- Publish to DockerHub requires the workflow to be run `manually` by providing `branch name` and `OAuth token`
- `OAuth Token` is generated after successful authentication against ASF OAuth, more details here: https://idm.apache.org/api.html
- Workflow checks for PMC level permissions to authorize publishing to DockerHub
- Workflow can be conveniently used from the release branch at the end of a release to publish images.
## How was this patch tested?
CI Run with successful build stage: https://github.com/kumaab/ranger/actions/runs/16375834642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab commented on PR #613:
URL: https://github.com/apache/ranger/pull/613#issuecomment-4017763992
This PR has been checked-in to apache/ranger-tools here: https://github.com/apache/ranger-tools/pull/8, so this is no longer required hence closing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry [ranger]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #613:
URL: https://github.com/apache/ranger/pull/613#discussion_r2219237185
##########
.github/workflows/build-and-tag-ranger-image.yml:
##########
@@ -0,0 +1,190 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-ranger-image
+
+# This workflow builds the following images: ranger, ranger-db, ranger-solr, ranger-zk.
+# It also pushes the image to the GitHub Container Registry, tagging it based on the ranger version present in the release branch.
+# It pushes the images to DockerHub if an OAuth token is provided as input.
+
+# Use this command to generate a unique 11 character token:
+# code=$(uuidgen | tr A-Z a-z | cut -c 1-11)
+# then pass the code to state param here: http://oauth.apache.org/auth?redirect_uri=https://github.com&state=code
+# On successful authentication, it generates an OAuth token on redirect_uri that can be used to trigger the workflow and push the images to DockerHub.
+
+# For more info, read ASF OAuth doc here: https://idm.apache.org/api.html
+on:
+ workflow_dispatch:
+ inputs:
+ token:
+ description: 'OAuth Access Token'
+ required: true
+ type: string
+ push:
+ branches:
+ - 'ranger-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Download build-8 artifacts
+ uses: dawidd6/action-download-artifact@v11
+ with:
+ name: target-8
+ workflow: ci.yml
+
+ - name: Copy artifacts for docker build
+ run: |
+ cp ranger-*-admin.tar.gz dev-support/ranger-docker/dist
+ cp version dev-support/ranger-docker/dist
+
+ - name: Run download-archives.sh
+ run: |
+ cd dev-support/ranger-docker
+ ./download-archives.sh none
+
+ - name: Generate image ID
+ id: meta
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ghcr.io/${{ github.repository_owner }}/ranger
+ tags: |
+ type=match,pattern=ranger-(.*),value={{branch}},group=1
+ flavor: |
+ latest=false
+
+ - name: Set up QEMU
+ uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
+
+ - name: Set up Docker Buildx
+ uses: docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349
+
+ - name: Login to GitHub Container Registry
+ id: login
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push images
+ id: build
+ working-directory: dev-support/ranger-docker
+ run: |
+ set -o allexport
+ source .env
+ set +o allexport
+ docker buildx build \
+ --build-arg RANGER_BASE_IMAGE=${RANGER_BASE_IMAGE} \
+ --build-arg RANGER_BASE_VERSION=${RANGER_BASE_VERSION} \
+ --build-arg RANGER_VERSION=${RANGER_VERSION} \
+ --build-arg RANGER_DB_TYPE=postgres \
+ --file Dockerfile.ranger \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger:${RANGER_VERSION} \
+ --push .
+ docker buildx build \
+ --build-arg POSTGRES_VERSION=${POSTGRES_VERSION} \
+ --file Dockerfile.ranger-postgres \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger-db:${RANGER_VERSION} \
+ --push .
+ docker buildx build \
+ --build-arg SOLR_VERSION=${SOLR_VERSION} \
+ --file Dockerfile.ranger-solr \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger-solr:${RANGER_VERSION} \
+ --push .
+ docker buildx build \
+ --build-arg ZK_VERSION=${ZK_VERSION} \
+ --file Dockerfile.ranger-zk \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger-zk:${RANGER_VERSION} \
+ --push .
+
+ tag:
+ needs: build
+ if: ${{ github.event.inputs.token != '' }}
+ runs-on: ubuntu-latest
+ env:
+ DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
+ IMAGE_ID: ${{ needs.build.outputs.image-id }}
Review Comment:
`build` job should define this output to allow `tag` to use it, something like:
https://github.com/apache/ozone-docker-runner/blob/1aa8dee3e97d870106b61736d78030e64ebe71b6/.github/workflows/build.yaml#L41-L42
Also, `image-id` will contain only `ranger` image, the other 3 images will not be tagged by this job.
BTW, in the long run, we should avoid building these 3 additional images.
- `zookeeper` image can be used directly.
- `postgres` and `solr` images can also be used, with Ranger-specific files (and password for Postgres) provided by users at runtime (in Docker Compose service definition).
In addition to simplifying the workflow, this would also avoid the problem of hard-coded password in the DB image (which makes it unusable for "production").
##########
.github/workflows/build-and-tag-ranger-image.yml:
##########
@@ -0,0 +1,190 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-ranger-image
+
+# This workflow builds the following images: ranger, ranger-db, ranger-solr, ranger-zk.
+# It also pushes the image to the GitHub Container Registry, tagging it based on the ranger version present in the release branch.
+# It pushes the images to DockerHub if an OAuth token is provided as input.
+
+# Use this command to generate a unique 11 character token:
+# code=$(uuidgen | tr A-Z a-z | cut -c 1-11)
+# then pass the code to state param here: http://oauth.apache.org/auth?redirect_uri=https://github.com&state=code
+# On successful authentication, it generates an OAuth token on redirect_uri that can be used to trigger the workflow and push the images to DockerHub.
+
+# For more info, read ASF OAuth doc here: https://idm.apache.org/api.html
+on:
+ workflow_dispatch:
+ inputs:
+ token:
+ description: 'OAuth Access Token'
+ required: true
+ type: string
+ push:
+ branches:
+ - 'ranger-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Download build-8 artifacts
+ uses: dawidd6/action-download-artifact@v11
+ with:
+ name: target-8
+ workflow: ci.yml
+
+ - name: Copy artifacts for docker build
+ run: |
+ cp ranger-*-admin.tar.gz dev-support/ranger-docker/dist
+ cp version dev-support/ranger-docker/dist
+
+ - name: Run download-archives.sh
+ run: |
+ cd dev-support/ranger-docker
+ ./download-archives.sh none
+
+ - name: Generate image ID
+ id: meta
+ uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+ with:
+ images: |
+ ghcr.io/${{ github.repository_owner }}/ranger
+ tags: |
+ type=match,pattern=ranger-(.*),value={{branch}},group=1
+ flavor: |
+ latest=false
+
+ - name: Set up QEMU
+ uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
+
+ - name: Set up Docker Buildx
+ uses: docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349
+
+ - name: Login to GitHub Container Registry
+ id: login
+ uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push images
+ id: build
+ working-directory: dev-support/ranger-docker
+ run: |
+ set -o allexport
+ source .env
+ set +o allexport
+ docker buildx build \
+ --build-arg RANGER_BASE_IMAGE=${RANGER_BASE_IMAGE} \
+ --build-arg RANGER_BASE_VERSION=${RANGER_BASE_VERSION} \
+ --build-arg RANGER_VERSION=${RANGER_VERSION} \
+ --build-arg RANGER_DB_TYPE=postgres \
+ --file Dockerfile.ranger \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger:${RANGER_VERSION} \
+ --push .
+ docker buildx build \
+ --build-arg POSTGRES_VERSION=${POSTGRES_VERSION} \
+ --file Dockerfile.ranger-postgres \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger-db:${RANGER_VERSION} \
+ --push .
+ docker buildx build \
+ --build-arg SOLR_VERSION=${SOLR_VERSION} \
+ --file Dockerfile.ranger-solr \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger-solr:${RANGER_VERSION} \
+ --push .
+ docker buildx build \
+ --build-arg ZK_VERSION=${ZK_VERSION} \
+ --file Dockerfile.ranger-zk \
+ --platform linux/amd64,linux/arm64 \
+ --tag ghcr.io/${{ github.repository_owner }}/ranger-zk:${RANGER_VERSION} \
+ --push .
+
+ tag:
+ needs: build
+ if: ${{ github.event.inputs.token != '' }}
+ runs-on: ubuntu-latest
+ env:
+ DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
+ IMAGE_ID: ${{ needs.build.outputs.image-id }}
+ REGISTRIES: ghcr.io # docker.io is appended dynamically
+ steps:
+ - name: Verify OAuth Token
+ run: |
+ response=$(curl https://oauth.apache.org/token\?code\=${{ github.event.inputs.token }})
+ echo "$response" | jq -e . >/dev/null 2>&1
Review Comment:
- `curl` will output download progress to std.out, `jq` will fail to parse it. Use `-LSs`
- Token shown in log in plain text.
##########
.github/workflows/build-and-tag-ranger-image.yml:
##########
@@ -0,0 +1,190 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-ranger-image
+
+# This workflow builds the following images: ranger, ranger-db, ranger-solr, ranger-zk.
+# It also pushes the image to the GitHub Container Registry, tagging it based on the ranger version present in the release branch.
+# It pushes the images to DockerHub if an OAuth token is provided as input.
+
+# Use this command to generate a unique 11 character token:
+# code=$(uuidgen | tr A-Z a-z | cut -c 1-11)
+# then pass the code to state param here: http://oauth.apache.org/auth?redirect_uri=https://github.com&state=code
+# On successful authentication, it generates an OAuth token on redirect_uri that can be used to trigger the workflow and push the images to DockerHub.
+
+# For more info, read ASF OAuth doc here: https://idm.apache.org/api.html
+on:
+ workflow_dispatch:
+ inputs:
+ token:
+ description: 'OAuth Access Token'
+ required: true
+ type: string
+ push:
+ branches:
+ - 'ranger-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Download build-8 artifacts
+ uses: dawidd6/action-download-artifact@v11
Review Comment:
Looks like this action is on Apache Infra [whitelist](https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml#L96), so this is OK.
##########
.github/workflows/build-and-tag-ranger-image.yml:
##########
@@ -0,0 +1,190 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: build-and-tag-ranger-image
+
+# This workflow builds the following images: ranger, ranger-db, ranger-solr, ranger-zk.
+# It also pushes the image to the GitHub Container Registry, tagging it based on the ranger version present in the release branch.
+# It pushes the images to DockerHub if an OAuth token is provided as input.
+
+# Use this command to generate a unique 11 character token:
+# code=$(uuidgen | tr A-Z a-z | cut -c 1-11)
+# then pass the code to state param here: http://oauth.apache.org/auth?redirect_uri=https://github.com&state=code
+# On successful authentication, it generates an OAuth token on redirect_uri that can be used to trigger the workflow and push the images to DockerHub.
+
+# For more info, read ASF OAuth doc here: https://idm.apache.org/api.html
+on:
+ workflow_dispatch:
+ inputs:
+ token:
+ description: 'OAuth Access Token'
+ required: true
+ type: string
+ push:
+ branches:
+ - 'ranger-**'
+
+permissions:
+ contents: read
+ packages: write
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Download build-8 artifacts
+ uses: dawidd6/action-download-artifact@v11
+ with:
+ name: target-8
+ workflow: ci.yml
+
+ - name: Copy artifacts for docker build
+ run: |
+ cp ranger-*-admin.tar.gz dev-support/ranger-docker/dist
+ cp version dev-support/ranger-docker/dist
Review Comment:
Why not `mv`, is the original file used later?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab closed pull request #613: RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry
URL: https://github.com/apache/ranger/pull/613
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab commented on PR #613:
URL: https://github.com/apache/ranger/pull/613#issuecomment-3090219220
@adoroszlai, Could you please review as well, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5246: Add workflow to build multi-platform ranger images and push to DockerHub and GitHub Container Registry [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab commented on PR #613:
URL: https://github.com/apache/ranger/pull/613#issuecomment-3794270985
Workflow run: https://github.com/kumaab/ranger/actions/runs/21312549198
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org