You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@zeppelin.apache.org by "rickcheng (Jira)" <ji...@apache.org> on 2021/07/23 06:45:00 UTC

[jira] [Created] (ZEPPELIN-5466) Can not specify the spark image from the interpreter settings

rickcheng created ZEPPELIN-5466:
-----------------------------------

             Summary: Can not specify the spark image from the interpreter settings
                 Key: ZEPPELIN-5466
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-5466
             Project: Zeppelin
          Issue Type: Bug
          Components: Kubernetes, spark
    Affects Versions: 0.9.0
            Reporter: rickcheng
            Assignee: rickcheng


When Zeppelin launches a spark interpreter pod under k8s mode, it will build the *spark submit options* through [buildSparkSubmitOptions()|[https://github.com/apache/zeppelin/blob/master/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java#L372-L393].]

 

 

But the property "spark.kubernetes.container.image" will use the value from the "

zConf.getK8sSparkContainerImage()

". Note that although 

the interpreter properties overrides the values through
{code:java}
// interpreter properties overrides the values
k8sProperties.putAll(Maps.fromProperties(properties));
{code}
 

But the spark submit options has already been built. So if user set the "spark.kubernetes.container.image" from the interpreter settings, it will not set the spark image in the spark submit options.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)