[zeppelin] branch branch-0.9 updated: [ZEPPELIN-5310] SPARK_HOME in zeppelin-env.sh doesn't take effect in yarn cluster mode

[zj...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zjffdu pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git


The following commit(s) were added to refs/heads/branch-0.9 by this push:
     new 006e88b  [ZEPPELIN-5310] SPARK_HOME in zeppelin-env.sh doesn't take effect in yarn cluster mode
006e88b is described below

commit 006e88b40fc27c5896233bd870b4356555a32a12
Author: Jeff Zhang <zj...@apache.org>
AuthorDate: Mon Apr 5 22:26:59 2021 +0800

    [ZEPPELIN-5310] SPARK_HOME in zeppelin-env.sh doesn't take effect in yarn cluster mode
    
    ### What is this PR for?
    
    The root cause is that we didn't get SPARK_HOME from env first when detecting scala version.
    
    ### What type of PR is it?
    [Bug Fix ]
    
    ### Todos
    * [ ] - Task
    
    ### What is the Jira issue?
    * https://issues.apache.org/jira/browse/ZEPPELIN-5310
    
    ### How should this be tested?
    * CI pass
    
    ### Screenshots (if appropriate)
    
    ### Questions:
    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? No
    
    Author: Jeff Zhang <zj...@apache.org>
    
    Closes #4088 from zjffdu/ZEPPELIN-5310 and squashes the following commits:
    
    8e916184a [Jeff Zhang] [ZEPPELIN-5310]. SPARK_HOME in zeppelin-env.sh doesn't take effect in yarn cluster mode
    
    (cherry picked from commit 73156375584fc368080fc2b78c85bc52bd8469d7)
    Signed-off-by: Jeff Zhang <zj...@apache.org>
---
 .../apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
index 489ed0d..e87ef81 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
@@ -124,7 +124,7 @@ public class SparkInterpreterLauncher extends StandardInterpreterLauncher {
           }
         }
 
-        String scalaVersion = detectSparkScalaVersion(properties.getProperty("SPARK_HOME"));
+        String scalaVersion = detectSparkScalaVersion(getEnv("SPARK_HOME"));
         Path scalaFolder =  Paths.get(zConf.getZeppelinHome(), "/interpreter/spark/scala-" + scalaVersion);
         if (!scalaFolder.toFile().exists()) {
           throw new IOException("spark scala folder " + scalaFolder.toFile() + " doesn't exist");