You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Michel Sumbul <mi...@yahoo.fr.INVALID> on 2020/06/26 12:53:00 UTC
Spark 3 pod template for the driver
Hi guys,
I try to use Spark 3 on top of Kubernetes and to specify a pod template for the driver.
Here is my pod manifest or the driver and when I do a spark-submit with the option:
--conf spark.kubernetes.driver.podTemplateFile=/data/k8s/podtemplate_driver3.yaml
I got the error message that I need to specify an image, but its the manifest.
Does my manifest file is wrong, how should it look like?
Thanks for your help,
Michel
--------The pod manifest:
apiVersion: v1kind: Podmetadata: name: mySpark3App labels: app: mySpark3App customlabel/app-id: "1"spec: securityContext: runAsUser: 1000 volumes: - name: "test-volume" emptyDir: {} containers: - name: spark3driver image: mydockerregistry.example.com/images/dev/spark3:latest instances: 1 resources: requests: cpu: "1000m" memory: "512Mi" limits: cpu: "1000m" memory: "512Mi" volumeMounts: - name: "test-volume" mountPath: "/tmp"