You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/10/13 09:53:04 UTC
[GitHub] [spark-docker] Yikun commented on a diff in pull request #9: [SPARK-40783][INFRA] Enable Spark on K8s integration test
Yikun commented on code in PR #9:
URL: https://github.com/apache/spark-docker/pull/9#discussion_r994426545
##########
.github/workflows/main.yml:
##########
@@ -76,29 +69,136 @@ jobs:
IMAGE_NAME=spark
IMAGE_PATH=${{ matrix.spark_version }}/$TAG
UNIQUE_IMAGE_TAG=${{ matrix.spark_version }}-$TAG
+ IMAGE_URL=$TEST_REPO/$IMAGE_NAME:$UNIQUE_IMAGE_TAG
- # Unique image tag in each version: scala2.12-java11-python3-ubuntu
+ # Unique image tag in each version: 3.3.0-scala2.12-java11-python3-ubuntu
echo "UNIQUE_IMAGE_TAG=${UNIQUE_IMAGE_TAG}" >> $GITHUB_ENV
# Test repo: ghcr.io/apache/spark-docker
echo "TEST_REPO=${TEST_REPO}" >> $GITHUB_ENV
# Image name: spark
echo "IMAGE_NAME=${IMAGE_NAME}" >> $GITHUB_ENV
# Image dockerfile path: 3.3.0/scala2.12-java11-python3-ubuntu
echo "IMAGE_PATH=${IMAGE_PATH}" >> $GITHUB_ENV
+ # Image URL: ghcr.io/apache/spark-docker/spark:3.3.0-scala2.12-java11-python3-ubuntu
+ echo "IMAGE_URL=${IMAGE_URL}" >> $GITHUB_ENV
- name: Print Image tags
run: |
echo "UNIQUE_IMAGE_TAG: "${UNIQUE_IMAGE_TAG}
echo "TEST_REPO: "${TEST_REPO}
echo "IMAGE_NAME: "${IMAGE_NAME}
echo "IMAGE_PATH: "${IMAGE_PATH}
+ echo "IMAGE_URL: "${IMAGE_URL}
- - name: Build and push test image
+ - name: Build image
uses: docker/build-push-action@v2
with:
context: ${{ env.IMAGE_PATH }}
- tags: ${{ env.TEST_REPO }}/${{ env.IMAGE_NAME }}:${{ env.UNIQUE_IMAGE_TAG }}
+ tags: ${{ env.IMAGE_URL }}
platforms: linux/amd64,linux/arm64
- - name: Image digest
- run: echo ${{ steps.docker_build.outputs.digest }}
+ - name: Test - Checkout Spark repository
+ uses: actions/checkout@v2
+ with:
+ fetch-depth: 0
+ repository: apache/spark
+ ref: v${{ matrix.spark_version }}
+ path: ${{ github.workspace }}/spark
+
+ - name: Test - Cherry pick commits
+ # Apache Spark enable resource limited k8s IT since v3.3.1, cherrpick patches for old release
+ # https://github.com/apache/spark/pull/36087#issuecomment-1251756266
+ if: matrix.spark_version == '3.3.0'
Review Comment:
Nope, it already in 3.3.1+ and 3.4.0+, so only 3.3.0 is enough!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org