[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

[GitBox <gi...@apache.org>]

dongjoon-hyun commented on a change in pull request #29846:
URL: https://github.com/apache/spark/pull/29846#discussion_r493881157



##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
##########
@@ -84,4 +112,15 @@ private[spark] class MountVolumesFeatureStep(conf: KubernetesConf)
       (volumeMount, volume)
     }
   }
+
+  override def getAdditionalKubernetesResources(): Seq[HasMetadata] = {
+    additionalResources
+  }
+}
+
+private[spark] object MountVolumesFeatureStep {
+  val PVC_ON_DEMAND = "OnDemand"
+  val PVC = "PersistentVolumeClaim"
+  val PVC_POSTFIX = "-pvc"
+  val PVC_ACCESS_MODE = "ReadWriteOnce"

Review comment:
       Ya, it's possible, but I didn't do that in this PR because of the followings.
   - `PVC_ON_DEMAND`: It's a dummy placeholder because the existing code expect some pre-defined names. We had better recommend a fixed one instead of making a configurable one in this case.
   - `PVC_POSTFIX`: It can be configurable but doesn't give much benefit because this is a part of transient ids.
   - `PVC_ACCESS_MODE`: Although this makes sense a lot, I leave this PR to focus on a fixed one because this PR aims to generate a new PVC for each executor. In other words, this PR is not suggesting creating a `ReadWriteMany` PVC and sharing across in multiple executors. 
   
   For `ReadWriteMany` PVC, we don't need to use this PR. The existing Spark PVC feature can mount a single `ReadWriteMany` PVC into all executors without any problem and there is no burden to maintain `ReadWriteMany` PVC, because it's always a single one. In addition, we also support NFS (AWS EFS) mounting, too.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org