You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yinan Li (JIRA)" <ji...@apache.org> on 2018/08/21 21:18:00 UTC
[jira] [Commented] (SPARK-25162) Kubernetes 'in-cluster' client
mode and value of spark.driver.host
[ https://issues.apache.org/jira/browse/SPARK-25162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588014#comment-16588014 ]
Yinan Li commented on SPARK-25162:
----------------------------------
We actually moved away from using the IP address of the driver pod to set {{spark.driver.host}}, to using a headless service to give the driver pod a FQDN name and set {{spark.driver.host}} to the FQDN name. Internally, we set {{spark.driver.bindAddress}} to the value of environment variable {{SPARK_DRIVER_BIND_ADDRESS}} which gets its value from the IP address of the pod using the downward API. We could do the same for {{spark.kubernetes.driver.pod.name}} as you suggested for in-cluster client mode.
> Kubernetes 'in-cluster' client mode and value of spark.driver.host
> ------------------------------------------------------------------
>
> Key: SPARK-25162
> URL: https://issues.apache.org/jira/browse/SPARK-25162
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.4.0
> Environment: A java program, deployed to kubernetes, that establishes a Spark Context in client mode.
> Not using spark-submit.
> Kubernetes 1.10
> AWS EKS
>
>
> Reporter: James Carter
> Priority: Minor
>
> When creating Kubernetes scheduler 'in-cluster' using client mode, the value for spark.driver.host can be derived from the IP address of the driver pod.
> I observed that the value of _spark.driver.host_ defaulted to the value of _spark.kubernetes.driver.pod.name_, which is not a valid hostname. This caused the executors to fail to establish a connection back to the driver.
> As a work around, in my configuration I pass the driver's pod name _and_ the driver's ip address to ensure that executors can establish a connection with the driver.
> _spark.kubernetes.driver.pod.name_ := env.valueFrom.fieldRef.fieldPath: metadata.name
> _spark.driver.host_ := env.valueFrom.fieldRef.fieldPath: status.podIp
> e.g.
> Deployment:
> {noformat}
> env:
> - name: DRIVER_POD_NAME
> valueFrom:
> fieldRef:
> fieldPath: metadata.name
> - name: DRIVER_POD_IP
> valueFrom:
> fieldRef:
> fieldPath: status.podIP
> {noformat}
>
> Application Properties:
> {noformat}
> config[spark.kubernetes.driver.pod.name]: ${DRIVER_POD_NAME}
> config[spark.driver.host]: ${DRIVER_POD_IP}
> {noformat}
>
> BasicExecutorFeatureStep.scala:
> {code:java}
> private val driverUrl = RpcEndpointAddress(
> kubernetesConf.get("spark.driver.host"),
> kubernetesConf.sparkConf.getInt("spark.driver.port", DEFAULT_DRIVER_PORT),
> CoarseGrainedSchedulerBackend.ENDPOINT_NAME).toString
> {code}
>
> Ideally only _spark.kubernetes.driver.pod.name_ would need be provided in this deployment scenario.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org