You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2021/07/23 07:21:38 UTC

[GitHub] [zeppelin] rickchengx opened a new pull request #4185: [ZEPPELIN-5466] Can not specify the spark image from the interpreter settings

rickchengx opened a new pull request #4185:
URL: https://github.com/apache/zeppelin/pull/4185


   
   
   ### What is this PR for?
   When Zeppelin launches a spark interpreter pod under k8s mode, it will build the **spark submit options** through [buildSparkSubmitOptions()](https://github.com/apache/zeppelin/blob/master/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java#L372-L393)
   
   But the property `--conf spark.kubernetes.container.image=` will use the value from the `zConf.getK8sSparkContainerImage()`. Note that although the interpreter properties overrides the values through
   
   ```
   // interpreter properties overrides the values
   k8sProperties.putAll(Maps.fromProperties(properties));
   ```
    
   But the spark submit options has already been built. So if user sets the `spark.kubernetes.container.image` from the interpreter settings, it will not correctly set the spark image in the `spark submit options`.
   
   
   ### What type of PR is it?
   [Bug Fix]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * <https://issues.apache.org/jira/browse/ZEPPELIN-5466> 
   
   ### How should this be tested?
   * CI pass and manually tested
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [zeppelin] zjffdu commented on pull request #4185: [ZEPPELIN-5466] Can not specify the spark image from the interpreter settings

Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4185:
URL: https://github.com/apache/zeppelin/pull/4185#issuecomment-891498802


   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [zeppelin] asfgit closed pull request #4185: [ZEPPELIN-5466] Can not specify the spark image from the interpreter settings

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4185:
URL: https://github.com/apache/zeppelin/pull/4185


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org