You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2020/04/01 06:37:47 UTC

[GitHub] [zeppelin] zjffdu opened a new pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url

zjffdu opened a new pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url
URL: https://github.com/apache/zeppelin/pull/3711
 
 
   ### What is this PR for?
    This is for injecting appId into custom spark url. Currently we allow user to set `zeppelin.spark.uiWebUrl` for a custom spark ui link. But we didn't inject appId into it. this make it less flexible for some cases. So this PR is to allow inject application id into custom spark url. e.g.
   `url_prefix/{{applicationId}}`
   
   
   ### What type of PR is it?
   [Improvement]
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/browse/ZEPPELIN-4710
   
   ### How should this be tested?
   * Unit test added
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zeppelin] alexott commented on a change in pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url

Posted by GitBox <gi...@apache.org>.
alexott commented on a change in pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url
URL: https://github.com/apache/zeppelin/pull/3711#discussion_r401481087
 
 

 ##########
 File path: spark/spark-scala-parent/src/main/scala/org/apache/zeppelin/spark/BaseSparkScalaInterpreter.scala
 ##########
 @@ -237,6 +237,13 @@ abstract class BaseSparkScalaInterpreter(val conf: SparkConf,
       case None =>
     }
 
+    val webUiUrl = properties.getProperty("zeppelin.spark.uiWebUrl");
+    if (!StringUtils.isBlank(webUiUrl)) {
+      this.sparkUrl = webUiUrl.replace("{{applicationId}}", sc.applicationId);
+    } else {
+      useYarnProxyURLIfNeeded()
+    }
+
 
 Review comment:
   I would move this into a separate function to remove duplicates.
   Another issue - we need to make safe value from `sc.applicationId`, so we couldn't get any security problem...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zeppelin] asfgit closed pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url
URL: https://github.com/apache/zeppelin/pull/3711
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zeppelin] zjffdu commented on a change in pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url

Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url
URL: https://github.com/apache/zeppelin/pull/3711#discussion_r404660422
 
 

 ##########
 File path: spark/spark-scala-parent/src/main/scala/org/apache/zeppelin/spark/BaseSparkScalaInterpreter.scala
 ##########
 @@ -237,6 +237,13 @@ abstract class BaseSparkScalaInterpreter(val conf: SparkConf,
       case None =>
     }
 
+    val webUiUrl = properties.getProperty("zeppelin.spark.uiWebUrl");
+    if (!StringUtils.isBlank(webUiUrl)) {
+      this.sparkUrl = webUiUrl.replace("{{applicationId}}", sc.applicationId);
+    } else {
+      useYarnProxyURLIfNeeded()
+    }
+
 
 Review comment:
   Thanks for the suggestion @alexott , I have removed the code duplicated. What do you mean security problem ? spark appId is generated by spark resource manager ? I don't see security problem

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zeppelin] alexott commented on a change in pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url

Posted by GitBox <gi...@apache.org>.
alexott commented on a change in pull request #3711: [ZEPPELIN-4710]. Allow to inject application id into custom spark url
URL: https://github.com/apache/zeppelin/pull/3711#discussion_r404664180
 
 

 ##########
 File path: spark/spark-scala-parent/src/main/scala/org/apache/zeppelin/spark/BaseSparkScalaInterpreter.scala
 ##########
 @@ -237,6 +237,13 @@ abstract class BaseSparkScalaInterpreter(val conf: SparkConf,
       case None =>
     }
 
+    val webUiUrl = properties.getProperty("zeppelin.spark.uiWebUrl");
+    if (!StringUtils.isBlank(webUiUrl)) {
+      this.sparkUrl = webUiUrl.replace("{{applicationId}}", sc.applicationId);
+    } else {
+      useYarnProxyURLIfNeeded()
+    }
+
 
 Review comment:
   Ok, that was my concern - if we can say that `applicationId` is safely generated, and not provided by user, then I'm ok...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services