You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2018/01/18 22:02:00 UTC
[jira] [Resolved] (SPARK-23133) Spark options are not passed to the
Executor in Docker context
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-23133.
------------------------------------
Resolution: Fixed
Fix Version/s: 2.3.0
Issue resolved by pull request 20322
[https://github.com/apache/spark/pull/20322]
> Spark options are not passed to the Executor in Docker context
> --------------------------------------------------------------
>
> Key: SPARK-23133
> URL: https://issues.apache.org/jira/browse/SPARK-23133
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.3.0
> Environment: Running Spark on K8s using supplied Docker image.
> Reporter: Andrew Korzhuev
> Priority: Minor
> Fix For: 2.3.0
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Reproduce:
> # Build image with `bin/docker-image-tool.sh`.
> # Submit application to k8s. Set executor options, e.g. ` --conf "spark.executor. extraJavaOptions=-Djava.security.auth.login.config=./jaas.conf"`
> # Visit Spark UI on executor and notice that option is not set.
> Expected behavior: options from spark-submit should be correctly passed to executor.
> Cause:
> `SPARK_EXECUTOR_JAVA_OPTS` is not defined in `entrypoint.sh`
> https://github.com/apache/spark/blob/branch-2.3/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh#L70
> [https://github.com/apache/spark/blob/branch-2.3/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh#L44-L45]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org