You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by zj...@apache.org on 2017/04/08 08:55:57 UTC
zeppelin git commit: ZEPPELIN-2150. NoSuchMethodError:
org.apache.spark.ui.SparkUI.appUIAddress() for the spark master
Repository: zeppelin
Updated Branches:
refs/heads/master 5381883ff -> 495be1ede
ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master
### What is this PR for?
`SparkUI.addUIAddress` is removed in spark master which cause this error. Actually spark 2.0 introduce new api `SparkContext.uiWebUrl` (SPARK-14576) which is public in contrast `SparkUI.addUIAddress` is private. This PR would use `SparkUI.addUIAddress` for spark 1 and `SparkContext.uiWebUrl` for spark 2.
### What type of PR is it?
[Improvement | Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2150
### How should this be tested?
Manually verify it on spark 1.6.2, spark 2.1.0 and spark master
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zj...@apache.org>
Closes #2231 from zjffdu/ZEPPELIN-2150 and squashes the following commits:
499b623 [Jeff Zhang] return null when sparkUIUrl is not avaialbe
e285278 [Jeff Zhang] ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/495be1ed
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/495be1ed
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/495be1ed
Branch: refs/heads/master
Commit: 495be1ede1c0f5fc949275153dcaa1c6de5ddcdd
Parents: 5381883
Author: Jeff Zhang <zj...@apache.org>
Authored: Sat Apr 8 08:15:14 2017 +0800
Committer: Jeff Zhang <zj...@apache.org>
Committed: Sat Apr 8 16:55:46 2017 +0800
----------------------------------------------------------------------
.../apache/zeppelin/spark/SparkInterpreter.java | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/495be1ed/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
----------------------------------------------------------------------
diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
index 6a2f3cc..af97d87 100644
--- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
+++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
@@ -999,10 +999,19 @@ public class SparkInterpreter extends Interpreter {
if (sparkUrl != null) {
return sparkUrl;
}
- Option<SparkUI> sparkUiOption = (Option<SparkUI>) Utils.invokeMethod(sc, "ui");
- SparkUI sparkUi = sparkUiOption.get();
- String sparkWebUrl = sparkUi.appUIAddress();
- return sparkWebUrl;
+
+ if (sparkVersion.newerThanEquals(SparkVersion.SPARK_2_0_0)) {
+ Option<String> uiWebUrlOption = (Option<String>) Utils.invokeMethod(sc, "uiWebUrl");
+ if (uiWebUrlOption.isDefined()) {
+ return uiWebUrlOption.get();
+ }
+ } else {
+ Option<SparkUI> sparkUIOption = (Option<SparkUI>) Utils.invokeMethod(sc, "ui");
+ if (sparkUIOption.isDefined()) {
+ return (String) Utils.invokeMethod(sparkUIOption.get(), "appUIAddress");
+ }
+ }
+ return null;
}
private Results.Result interpret(String line) {