You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/11/05 16:57:00 UTC
[jira] [Updated] (SPARK-47010) Kubernetes: support csi driver for volume type
[ https://issues.apache.org/jira/browse/SPARK-47010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated SPARK-47010:
-----------------------------------
Labels: pull-request-available (was: )
> Kubernetes: support csi driver for volume type
> ----------------------------------------------
>
> Key: SPARK-47010
> URL: https://issues.apache.org/jira/browse/SPARK-47010
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 3.5.0
> Reporter: Oleg Frenkel
> Priority: Major
> Labels: pull-request-available
>
> Today Spark supports the following types of Kubernetes [volumes|https://kubernetes.io/docs/concepts/storage/volumes/]: hostPath, emptyDir, nfs and persistentVolumeClaim.
> In our case, Kubernetes cluster is multi-tenant and we cannot make cluster-wide changes when deploying our application to the Kubernetes cluster. Our application requires static shared file system. So, we cannot use hostPath (don't have control of hosting VMs) and persistentVolumeClaim (requires cluster-wide change when deploying PV). Our security department does not allow nfs.
> What would help in our case, is the use of csi driver (taken from here: https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/deploy/example/e2e_usage.md#option3-inline-volume):
> {code:java}
> kind: Pod
> apiVersion: v1
> metadata:
> name: nginx-azurefile-inline-volume
> spec:
> nodeSelector:
> "kubernetes.io/os": linux
> containers:
> - image: mcr.microsoft.com/oss/nginx/nginx:1.19.5
> name: nginx-azurefile
> command:
> - "/bin/bash"
> - "-c"
> - set -euo pipefail; while true; do echo $(date) >> /mnt/azurefile/outfile; sleep 1; done
> volumeMounts:
> - name: persistent-storage
> mountPath: "/mnt/azurefile"
> readOnly: false
> volumes:
> - name: persistent-storage
> csi:
> driver: file.csi.azure.com
> volumeAttributes:
> shareName: EXISTING_SHARE_NAME # required
> secretName: azure-secret # required
> mountOptions: "dir_mode=0777,file_mode=0777,cache=strict,actimeo=30,nosharesock" # optional {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org