You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hu Ziqian (Jira)" <ji...@apache.org> on 2023/03/10 06:29:00 UTC

[jira] [Created] (SPARK-42742) access apiserver by pod env

Hu Ziqian created SPARK-42742:
---------------------------------

             Summary: access apiserver by pod env
                 Key: SPARK-42742
                 URL: https://issues.apache.org/jira/browse/SPARK-42742
             Project: Spark
          Issue Type: Bug
          Components: Kubernetes
    Affects Versions: 3.1.2
            Reporter: Hu Ziqian


When start spark on k8s，driver pod  use spark.kubernetes.driver.master to get apiserver address. This config  us  [https://kubernetes.default.svc|https://kubernetes.default.svc/] as default and do not care about the apiserver port.

In our case, apiserver port is not 443 will driver will throw connectException. As k8s doc mentioned （https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/#directly-accessing-the-rest-api）, we can get master url by getting {{KUBERNETES_SERVICE_HOST}} and {{KUBERNETES_SERVICE_PORT_HTTPS}} environment variables from pod. So we add a new conf spark.kubernetes.driver.master.from.pod.env to allow driver get master url from env in cluster mode on k8s



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org