You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/02/07 07:35:00 UTC
[spark] branch branch-3.4 updated: [SPARK-42368][INFRA][TESTS] Exclude SparkRemoteFileTest from GitHub Action K8s IT job
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new bf4722282ca [SPARK-42368][INFRA][TESTS] Exclude SparkRemoteFileTest from GitHub Action K8s IT job
bf4722282ca is described below
commit bf4722282ca14fa164e311e92664204c789edc2d
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Feb 6 23:34:31 2023 -0800
[SPARK-42368][INFRA][TESTS] Exclude SparkRemoteFileTest from GitHub Action K8s IT job
### What changes were proposed in this pull request?
This PR aims to exclude `SparkRemoteFileTest` from K8s IT GitHub Action job.
```
$ git grep localTestTag
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/BasicTestsSuite.scala: import KubernetesSuite.{k8sTestTag, localTestTag}
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/BasicTestsSuite.scala: test("Run SparkRemoteFileTest using a remote data file", k8sTestTag, localTestTag) {
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala: val localTestTag = Tag("local")
```
### Why are the changes needed?
Since #39697 , I've been monitoring this job. The test is too flaky in GitHub Action environment due to the limited resources and slowness.
**MASTER**
- https://github.com/apache/spark/actions/runs/4110706261/jobs/7093797830
- https://github.com/apache/spark/actions/runs/4110355193/jobs/7093093588
**BRANCH-3.4**
- https://github.com/apache/spark/actions/runs/4110731960/jobs/7093844949
### Does this PR introduce _any_ user-facing change?
No. This is a test infra PR.
### How was this patch tested?
Pass the CIs.
Closes #39921 from dongjoon-hyun/SPARK-42368.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 8998a3b5c8b43d15000ddfa04bea568d8b3ffb54)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.github/workflows/build_and_test.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index 03e76f5fed7..8dd3f498261 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -963,7 +963,7 @@ jobs:
kubectl create clusterrolebinding serviceaccounts-cluster-admin --clusterrole=cluster-admin --group=system:serviceaccounts || true
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/v1.7.0/installer/volcano-development.yaml || true
eval $(minikube docker-env)
- build/sbt -Psparkr -Pkubernetes -Pvolcano -Pkubernetes-integration-tests -Dspark.kubernetes.test.driverRequestCores=0.5 -Dspark.kubernetes.test.executorRequestCores=0.2 -Dspark.kubernetes.test.volcanoMaxConcurrencyJobNum=1 "kubernetes-integration-tests/test"
+ build/sbt -Psparkr -Pkubernetes -Pvolcano -Pkubernetes-integration-tests -Dspark.kubernetes.test.driverRequestCores=0.5 -Dspark.kubernetes.test.executorRequestCores=0.2 -Dspark.kubernetes.test.volcanoMaxConcurrencyJobNum=1 -Dtest.exclude.tags=local "kubernetes-integration-tests/test"
- name: Upload Spark on K8S integration tests log files
if: failure()
uses: actions/upload-artifact@v3
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org