You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Dongqing Liu (JIRA)" <ji...@apache.org> on 2018/12/19 07:48:00 UTC

[jira] [Created] (SPARK-26404) set spark.pyspark.python or PYSPARK_PYTHON doesn't work in k8s client-cluster mode.

Dongqing  Liu created SPARK-26404:
-------------------------------------

             Summary: set spark.pyspark.python or PYSPARK_PYTHON doesn't work in k8s client-cluster mode.
                 Key: SPARK-26404
                 URL: https://issues.apache.org/jira/browse/SPARK-26404
             Project: Spark
          Issue Type: Bug
          Components: Kubernetes
    Affects Versions: 2.4.0
            Reporter: Dongqing  Liu


Neither

   conf.set("spark.executorEnv.PYSPARK_PYTHON", "/opt/pythonenvs/bin/python")

nor 

  conf.set("spark.pyspark.python", "/opt/pythonenvs/bin/python") 

works. 

Looks like the executor always picks python from PATH.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org