You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2024/01/21 01:51:23 UTC
(spark) branch master updated: [SPARK-46786][K8S] Fix `MountVolumesFeatureStep` to use `ReadWriteOncePod` instead of `ReadWriteOnce`
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 45ec74415a4a [SPARK-46786][K8S] Fix `MountVolumesFeatureStep` to use `ReadWriteOncePod` instead of `ReadWriteOnce`
45ec74415a4a is described below
commit 45ec74415a4a89851968941b80c490e37ee88daf
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Sat Jan 20 17:51:12 2024 -0800
[SPARK-46786][K8S] Fix `MountVolumesFeatureStep` to use `ReadWriteOncePod` instead of `ReadWriteOnce`
### What changes were proposed in this pull request?
This PR aims to fix a duplicated volume mounting bug by using `ReadWriteOncePod` instead of `ReadWriteOnce`.
### Why are the changes needed?
This bug fix is based on the stable K8s feature which is available since v1.22.
- [KEP-2485: ReadWriteOncePod PersistentVolume AccessMode](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode/README.md)
- https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
- v1.22 Alpha
- v1.27 Beta
- v1.29 Stable
For the record, the minimum K8s version of GKE/EKS/AKE is **v1.24** as of today and the latest v1.29 is supported like the following.
- [2024.01 (GKE Regular Channel)](https://cloud.google.com/kubernetes-engine/docs/release-schedule)
- [2024.02 (AKE GA)](https://learn.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar)
### Does this PR introduce _any_ user-facing change?
This is a bug fix.
### How was this patch tested?
Pass the CIs with the existing PV-related tests.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #44817 from dongjoon-hyun/SPARK-46786.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala | 2 +-
.../org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
index 031d18545b89..603e24f745cd 100644
--- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
+++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
@@ -126,5 +126,5 @@ private[spark] object MountVolumesFeatureStep {
val PVC_ON_DEMAND = "OnDemand"
val PVC = "PersistentVolumeClaim"
val PVC_POSTFIX = "-pvc"
- val PVC_ACCESS_MODE = "ReadWriteOnce"
+ val PVC_ACCESS_MODE = "ReadWriteOncePod"
}
diff --git a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
index 1b5a25306671..212eef0d9ac2 100644
--- a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
+++ b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
@@ -136,7 +136,7 @@ private[spark] trait PVTestsSuite { k8sSuite: KubernetesSuite =>
}
}
- test("PVs with local hostpath and storageClass on statefulsets", k8sTestTag, pvTestTag) {
+ ignore("PVs with local hostpath and storageClass on statefulsets", k8sTestTag, pvTestTag) {
assume(this.getClass.getSimpleName == "KubernetesSuite")
sparkAppConf
.set(s"spark.kubernetes.driver.volumes.persistentVolumeClaim.data.mount.path",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org