You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2021/07/15 07:05:37 UTC
[GitHub] [zeppelin] rickchengx opened a new pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
rickchengx opened a new pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176
### What is this PR for?
Currently, when zeppelin server is running outside of k8s cluster, users cannot access the spark UI directly from the web page of Zeppelin. So this PR creates ingress resource in `k8s/interpreter/100-interpreter-spec.yaml` and enables the `SPARK JOB` button to jump directly to the URL defined by ingress.
If user does not specify the property `zeppelin.spark.uiWebUrl`, it will be automatically generated according to the template: `{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}`, which is also the URL that the `SPARK JOB` button points to. So this PR also configured the ingress.host by this URL.
This PR also adds a property `zeppelin.k8s.spark.useIngress` to let user choose whether to create ingress.
Example usage:
```
zeppelin.k8s.spark.useIngress true
```
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* <https://issues.apache.org/jira/browse/ZEPPELIN-5445>
### How should this be tested?
* CI pass and manually tested
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on a change in pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
Posted by GitBox <gi...@apache.org>.
Reamer commented on a change in pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176#discussion_r670992722
##########
File path: zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
##########
@@ -307,15 +307,19 @@ Properties getTemplateBindings(String userName) {
// configure interpreter property "zeppelin.spark.uiWebUrl" if not defined, to enable spark ui through reverse proxy
String webUrl = (String) properties.get("zeppelin.spark.uiWebUrl");
if (StringUtils.isBlank(webUrl)) {
- webUrl = "//{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}";
+ webUrl = "{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}";
+ String bindedWebUrl = sparkUiWebUrlFromTemplate(
+ webUrl,
+ webUiPort,
+ getPodName(),
+ k8sEnv.get(ENV_SERVICE_DOMAIN)
+ );
+ properties.put("zeppelin.spark.uiWebUrl", "//" + bindedWebUrl);
+
+ //configure the ingress.host to binded web url, to enable spark ui through ingress
+ k8sProperties.put("zeppelin.k8s.spark.ingress.host", bindedWebUrl);
}
- properties.put("zeppelin.spark.uiWebUrl",
Review comment:
Why are you moving this part to the `if`?
In my environment I have defined a custom `webUrl` and I need the function `SparkUiWebUrlFromTemplate`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] asfgit closed pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] rickchengx commented on pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
Posted by GitBox <gi...@apache.org>.
rickchengx commented on pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176#issuecomment-881265809
When user does not specify the property `zeppelin.spark.uiWebUrl`, it will be automatically `//{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}`. And the default value of `zeppelin.k8s.spark.ingress.host` is `{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}`.
So if user does not customize the URL, he just needs to set the property `zeppelin.k8s.spark.useIngress true` to create the ingress. And he can directly click the button `SPARK JOB` to jump to the spark UI through ingress.
But if user customizes the URL, he can also set the property `zeppelin.k8s.spark.ingress.host` to enable the spark UI through ingress :
```
zeppelin.spark.uiWebUrl //{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}
zeppelin.k8s.spark.useIngress true
zeppelin.k8s.spark.ingress.host {{SERVICE_NAME}}.{{SERVICE_DOMAIN}}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] rickchengx commented on pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
Posted by GitBox <gi...@apache.org>.
rickchengx commented on pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176#issuecomment-883285849
> Please add documentation. Everything else looks good.
Thanks for the comment. The documentation has been added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] rickchengx commented on pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
Posted by GitBox <gi...@apache.org>.
rickchengx commented on pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176#issuecomment-883285849
> Please add documentation. Everything else looks good.
Thanks for the comment. The documentation has been added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] rickchengx commented on a change in pull request #4176: [ZEPPELIN-5445] Allow users to access the spark UI outside the kubernetes cluster
Posted by GitBox <gi...@apache.org>.
rickchengx commented on a change in pull request #4176:
URL: https://github.com/apache/zeppelin/pull/4176#discussion_r671050695
##########
File path: zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
##########
@@ -307,15 +307,19 @@ Properties getTemplateBindings(String userName) {
// configure interpreter property "zeppelin.spark.uiWebUrl" if not defined, to enable spark ui through reverse proxy
String webUrl = (String) properties.get("zeppelin.spark.uiWebUrl");
if (StringUtils.isBlank(webUrl)) {
- webUrl = "//{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}";
+ webUrl = "{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}";
+ String bindedWebUrl = sparkUiWebUrlFromTemplate(
+ webUrl,
+ webUiPort,
+ getPodName(),
+ k8sEnv.get(ENV_SERVICE_DOMAIN)
+ );
+ properties.put("zeppelin.spark.uiWebUrl", "//" + bindedWebUrl);
+
+ //configure the ingress.host to binded web url, to enable spark ui through ingress
+ k8sProperties.put("zeppelin.k8s.spark.ingress.host", bindedWebUrl);
}
- properties.put("zeppelin.spark.uiWebUrl",
Review comment:
> Why are you moving this part to the `if`?
> In my environment I have defined a custom `webUrl` and I need the function `SparkUiWebUrlFromTemplate`.
Thanks for the comment. I realized that this is a mistake since the customed `zeppelin.spark.uiWebUrl` also needs to be bound by the function `SparkUiWebUrlFromTemplate`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org