You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hu Ziqian (Jira)" <ji...@apache.org> on 2023/03/10 06:58:00 UTC
[jira] [Commented] (SPARK-42742) access apiserver by pod env
[ https://issues.apache.org/jira/browse/SPARK-42742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698760#comment-17698760 ]
Hu Ziqian commented on SPARK-42742:
-----------------------------------
Created PR [https://github.com/apache/spark/pull/40361] for this issue.
> access apiserver by pod env
> ---------------------------
>
> Key: SPARK-42742
> URL: https://issues.apache.org/jira/browse/SPARK-42742
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 3.1.2
> Reporter: Hu Ziqian
> Priority: Major
>
> When start spark on k8s，driver pod use spark.kubernetes.driver.master to get apiserver address. This config us [https://kubernetes.default.svc|https://kubernetes.default.svc/] as default and do not care about the apiserver port.
> In our case, apiserver port is not 443 will driver will throw connectException. As k8s doc mentioned （https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/#directly-accessing-the-rest-api）, we can get master url by getting {{KUBERNETES_SERVICE_HOST}} and {{KUBERNETES_SERVICE_PORT_HTTPS}} environment variables from pod. So we add a new conf spark.kubernetes.driver.master.from.pod.env to allow driver get master url from env in cluster mode on k8s
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org