You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Michel Sumbul <mi...@yahoo.fr.INVALID> on 2020/06/26 12:53:00 UTC

Spark 3 pod template for the driver

Hi guys,
I try to use Spark 3 on top of Kubernetes and to specify a pod template for the driver.
Here is my pod manifest or the driver and when I do a spark-submit with the option:
--conf spark.kubernetes.driver.podTemplateFile=/data/k8s/podtemplate_driver3.yaml

I got the error message that I need to specify an image, but its the manifest.

Does my manifest file is wrong, how should it look like?

Thanks for your help,
Michel
--------The pod manifest:
apiVersion: v1kind: Podmetadata:  name: mySpark3App  labels:    app: mySpark3App    customlabel/app-id: "1"spec:  securityContext:    runAsUser: 1000  volumes:    - name: "test-volume"      emptyDir: {}  containers:    - name: spark3driver      image: mydockerregistry.example.com/images/dev/spark3:latest      instances: 1      resources:        requests:          cpu: "1000m"          memory: "512Mi"        limits:          cpu: "1000m"          memory: "512Mi"      volumeMounts:       - name: "test-volume"         mountPath: "/tmp"