You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mi...@apache.org on 2016/09/23 07:47:36 UTC
zeppelin git commit: [ZEPPELIN-1026] set syntax highlight based on
default bound interpreter
Repository: zeppelin
Updated Branches:
refs/heads/master 6f9012b40 -> a3ca80031
[ZEPPELIN-1026] set syntax highlight based on default bound interpreter
### What is this PR for?
This is complete work of #1148. Comments and tasks on #1148 has been handled in this PR.
- Add syntax language information in `interpreter-setting.json`
- When user type `%replName` in paragraph, back-end check if the interpreter name with `replName` exists, and return language information to front-end if it does
- If user doesn't specify `%replName`, default interpreter's language will be used
- Using alias name for paragraph syntax highlight
### What type of PR is it?
[Bug Fix | Improvement]
### What is the Jira issue?
[ZEPPELIN-1026](https://issues.apache.org/jira/browse/ZEPPELIN-1026)
### How should this be tested?
1. Create new note and make markdown interpreter to be default.
2. See if markdown syntax is applied.
### Screenshots (if appropriate)
#### Case 1. When the default interpreter set to python interpreter.
**Before**
Has `scala` as syntax highlight language when %python is not set.
<img width="665" alt="screen shot 2016-07-07 at 10 46 20 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png">
**After**
Has `python` as syntax highlight language even when %python is not set.
<img width="666" alt="screen shot 2016-07-07 at 10 44 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png">
#### Case 2. When use alias name as repl name.
**Before**
<img width="742" alt="screen shot 2016-09-08 at 4 22 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png">
**After**
<img width="741" alt="screen shot 2016-09-08 at 4 34 57 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png">
### Further possible improvements
There are still several cases that Zeppelin doesn't handle syntax highlight well. These can be handled with another jira ticket/PR.
1. When default bound interpreter changes, syntax highlight is not changed accordingly
2. When copy/paste code, syntax highlight won't be applied properly since Zeppelin only checks changes when cursor is in first line.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes(for creating new interpreter)
Author: Mina Lee <mi...@apache.org>
Closes #1415 from minahlee/ZEPPELIN-1026 and squashes the following commits:
c66fb0e [Mina Lee] Move getEditorSetting to InterpreterFactory class
2d56222 [Mina Lee] Add description about default syntax highlight in doc
08ccad9 [Mina Lee] Fix test
0874522 [Mina Lee] Change condition for triggering 'getAndSetEditorSetting' to reduce front-end <-> back-end communication
9e4f2e9 [Mina Lee] Change the way to read interpreter language from interpreter-setting.json after #1145
75543b3 [Mina Lee] Add test
565d9d0 [Mina Lee] [DOC] Setting syntax highlight when writing new interpreter
20132ca [Mina Lee] Get paragraph editor mode from backend
52f4207 [Mina Lee] Align comments for readability
26cbbb8 [Mina Lee] Add editor field
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/a3ca8003
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/a3ca8003
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/a3ca8003
Branch: refs/heads/master
Commit: a3ca80031121a495f2946bd6209710dc7b6e330d
Parents: 6f9012b
Author: Mina Lee <mi...@apache.org>
Authored: Mon Sep 19 19:05:39 2016 +0200
Committer: Mina Lee <mi...@apache.org>
Committed: Fri Sep 23 16:47:26 2016 +0900
----------------------------------------------------------------------
.../src/main/resources/interpreter-setting.json | 3 +
docs/development/writingzeppelininterpreter.md | 20 ++-
.../src/main/resources/interpreter-setting.json | 3 +
.../src/main/resources/interpreter-setting.json | 3 +
.../src/main/resources/interpreter-setting.json | 12 ++
.../src/main/resources/interpreter-setting.json | 3 +
.../src/main/resources/interpreter-setting.json | 3 +
.../src/main/resources/interpreter-setting.json | 5 +-
.../src/main/resources/interpreter-setting.json | 12 ++
.../sparkr-resources/interpreter-setting.json | 15 ++
.../zeppelin/interpreter/Interpreter.java | 6 +
.../apache/zeppelin/socket/NotebookServer.java | 21 ++-
zeppelin-web/bower.json | 4 +-
.../notebook/paragraph/paragraph.controller.js | 96 +++++++------
.../websocketEvents/websocketEvents.factory.js | 6 +-
.../websocketEvents/websocketMsg.service.js | 9 ++
.../interpreter/InterpreterFactory.java | 68 +++++++--
.../zeppelin/interpreter/InterpreterInfo.java | 15 +-
.../zeppelin/notebook/socket/Message.java | 143 ++++++++++---------
.../interpreter/InterpreterFactoryTest.java | 60 +++++++-
.../org/apache/zeppelin/notebook/NoteTest.java | 2 -
.../apache/zeppelin/notebook/NotebookTest.java | 34 ++---
.../interpreter/mock/interpreter-setting.json | 12 ++
23 files changed, 398 insertions(+), 157 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/bigquery/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/bigquery/src/main/resources/interpreter-setting.json b/bigquery/src/main/resources/interpreter-setting.json
index 3e524ed..fb44063 100644
--- a/bigquery/src/main/resources/interpreter-setting.json
+++ b/bigquery/src/main/resources/interpreter-setting.json
@@ -22,6 +22,9 @@
"defaultValue": "100000",
"description": "Maximum number of rows to fetch from BigQuery"
}
+ },
+ "editor": {
+ "language": "sql"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/docs/development/writingzeppelininterpreter.md
----------------------------------------------------------------------
diff --git a/docs/development/writingzeppelininterpreter.md b/docs/development/writingzeppelininterpreter.md
index 4ae1cbc..acf72e8 100644
--- a/docs/development/writingzeppelininterpreter.md
+++ b/docs/development/writingzeppelininterpreter.md
@@ -48,7 +48,7 @@ There are three locations where you can store your interpreter group, name and o
{ZEPPELIN_INTERPRETER_DIR}/{YOUR_OWN_INTERPRETER_DIR}/interpreter-setting.json
```
-Here is an example of `interpreter-setting.json` on your own interpreter.
+Here is an example of `interpreter-setting.json` on your own interpreter. Note that if you don't specify editor object, your interpreter will use plain text mode for syntax highlighting.
```json
[
@@ -69,6 +69,9 @@ Here is an example of `interpreter-setting.json` on your own interpreter.
"defaultValue": "property2DefaultValue",
"description": "Property 2 description"
}, ...
+ },
+ "editor": {
+ "language": "your-syntax-highlight-language"
}
},
{
@@ -81,8 +84,8 @@ Finally, Zeppelin uses static initialization with the following:
```
static {
- Interpreter.register("MyInterpreterName", MyClassName.class.getName());
- }
+ Interpreter.register("MyInterpreterName", MyClassName.class.getName());
+}
```
**Static initialization is deprecated and will be supported until 0.6.0.**
@@ -96,15 +99,20 @@ some interpreter specific code...
```
## Programming Languages for Interpreter
-If the interpreter uses a specific programming language ( like Scala, Python, SQL ), it is generally recommended to add a syntax highlighting supported for that to the notebook paragraph editor.
+If the interpreter uses a specific programming language (like Scala, Python, SQL), it is generally recommended to add a syntax highlighting supported for that to the notebook paragraph editor.
To check out the list of languages supported, see the `mode-*.js` files under `zeppelin-web/bower_components/ace-builds/src-noconflict` or from [github.com/ajaxorg/ace-builds](https://github.com/ajaxorg/ace-builds/tree/master/src-noconflict).
If you want to add a new set of syntax highlighting,
1. Add the `mode-*.js` file to <code>[zeppelin-web/bower.json](https://github.com/apache/zeppelin/blob/master/zeppelin-web/bower.json)</code> ( when built, <code>[zeppelin-web/src/index.html](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/index.html)</code> will be changed automatically. ).
-2. Add to the list of `editorMode` in <code>[zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js)</code> - it follows the pattern 'ace/mode/x' where x is the name.
-3. Add to the code that checks for `%` prefix and calls `session.setMode(editorMode.x)` in `setParagraphMode` located in <code>[zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js)</code>.
+2. Add `editor` object to `interpreter-setting.json` file. If you want to set your language to `java` for example, add:
+
+ ```
+ "editor": {
+ "language": "java"
+ }
+ ```
## Install your interpreter binary
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/flink/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/flink/src/main/resources/interpreter-setting.json b/flink/src/main/resources/interpreter-setting.json
index 067e7b8..e9bd126 100644
--- a/flink/src/main/resources/interpreter-setting.json
+++ b/flink/src/main/resources/interpreter-setting.json
@@ -16,6 +16,9 @@
"defaultValue": "6123",
"description": "port of running JobManager."
}
+ },
+ "editor": {
+ "language": "scala"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/jdbc/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/jdbc/src/main/resources/interpreter-setting.json b/jdbc/src/main/resources/interpreter-setting.json
index 289d4f8..abdb719 100644
--- a/jdbc/src/main/resources/interpreter-setting.json
+++ b/jdbc/src/main/resources/interpreter-setting.json
@@ -154,6 +154,9 @@
"defaultValue": "org.postgresql.Driver",
"description": ""
}
+ },
+ "editor": {
+ "language": "sql"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/livy/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/livy/src/main/resources/interpreter-setting.json b/livy/src/main/resources/interpreter-setting.json
index bc5a275..bad6830 100644
--- a/livy/src/main/resources/interpreter-setting.json
+++ b/livy/src/main/resources/interpreter-setting.json
@@ -87,6 +87,9 @@
"defaultValue": "",
"description": "Adding extra libraries to livy interpreter"
}
+ },
+ "editor": {
+ "language": "scala"
}
},
{
@@ -105,6 +108,9 @@
"defaultValue": "false",
"description": "Execute multiple SQL concurrently if set true."
}
+ },
+ "editor": {
+ "language": "sql"
}
},
{
@@ -112,6 +118,9 @@
"name": "pyspark",
"className": "org.apache.zeppelin.livy.LivyPySparkInterpreter",
"properties": {
+ },
+ "editor": {
+ "language": "python"
}
},
{
@@ -119,6 +128,9 @@
"name": "sparkr",
"className": "org.apache.zeppelin.livy.LivySparkRInterpreter",
"properties": {
+ },
+ "editor": {
+ "language": "r"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/markdown/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/markdown/src/main/resources/interpreter-setting.json b/markdown/src/main/resources/interpreter-setting.json
index 78ad735..38a6a76 100644
--- a/markdown/src/main/resources/interpreter-setting.json
+++ b/markdown/src/main/resources/interpreter-setting.json
@@ -10,6 +10,9 @@
"defaultValue": "markdown4j",
"description": "Markdown Parser Type. Available values: markdown4j, pegdown. Default = markdown4j"
}
+ },
+ "editor": {
+ "language": "markdown"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/python/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/python/src/main/resources/interpreter-setting.json b/python/src/main/resources/interpreter-setting.json
index 4b12cad..d3df586 100644
--- a/python/src/main/resources/interpreter-setting.json
+++ b/python/src/main/resources/interpreter-setting.json
@@ -16,6 +16,9 @@
"defaultValue": "1000",
"description": "Max number of dataframe rows to display."
}
+ },
+ "editor": {
+ "language": "python"
}
},
{
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/shell/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/shell/src/main/resources/interpreter-setting.json b/shell/src/main/resources/interpreter-setting.json
index 78621df..d1996fa 100644
--- a/shell/src/main/resources/interpreter-setting.json
+++ b/shell/src/main/resources/interpreter-setting.json
@@ -28,6 +28,9 @@
"defaultValue": "",
"description": "Kerberos principal"
}
+ },
+ "editor": {
+ "language": "sh"
}
}
-]
\ No newline at end of file
+]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/spark/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/spark/src/main/resources/interpreter-setting.json b/spark/src/main/resources/interpreter-setting.json
index d87a6c7..55d25bd 100644
--- a/spark/src/main/resources/interpreter-setting.json
+++ b/spark/src/main/resources/interpreter-setting.json
@@ -54,6 +54,9 @@
"defaultValue": "local[*]",
"description": "Spark master uri. ex) spark://masterhost:7077"
}
+ },
+ "editor": {
+ "language": "scala"
}
},
{
@@ -85,6 +88,9 @@
"defaultValue": "true",
"description": "Import implicits, UDF collection, and sql if set true. true by default."
}
+ },
+ "editor": {
+ "language": "sql"
}
},
{
@@ -104,6 +110,9 @@
"defaultValue": "spark-packages,http://dl.bintray.com/spark-packages/maven,false;",
"description": "A list of 'id,remote-repository-URL,is-snapshot;' for each remote repository."
}
+ },
+ "editor": {
+ "language": "scala"
}
},
{
@@ -117,6 +126,9 @@
"defaultValue": "python",
"description": "Python command to run pyspark with"
}
+ },
+ "editor": {
+ "language": "python"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/spark/src/main/sparkr-resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/spark/src/main/sparkr-resources/interpreter-setting.json b/spark/src/main/sparkr-resources/interpreter-setting.json
index f884fe4..338c861 100644
--- a/spark/src/main/sparkr-resources/interpreter-setting.json
+++ b/spark/src/main/sparkr-resources/interpreter-setting.json
@@ -54,6 +54,9 @@
"defaultValue": "local[*]",
"description": "Spark master uri. ex) spark://masterhost:7077"
}
+ },
+ "editor": {
+ "language": "scala"
}
},
{
@@ -85,6 +88,9 @@
"defaultValue": "true",
"description": "Import implicits, UDF collection, and sql if set true. true by default."
}
+ },
+ "editor": {
+ "language": "sql"
}
},
{
@@ -104,6 +110,9 @@
"defaultValue": "spark-packages,http://dl.bintray.com/spark-packages/maven,false;",
"description": "A list of 'id,remote-repository-URL,is-snapshot;' for each remote repository."
}
+ },
+ "editor": {
+ "language": "scala"
}
},
{
@@ -117,6 +126,9 @@
"defaultValue": "python",
"description": "Python command to run pyspark with"
}
+ },
+ "editor": {
+ "language": "python"
}
},
{
@@ -148,6 +160,9 @@
"defaultValue": "out.format = 'html', comment = NA, echo = FALSE, results = 'asis', message = F, warning = F",
"description": ""
}
+ },
+ "editor": {
+ "language": "r"
}
}
]
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
----------------------------------------------------------------------
diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
index 6d7d660..9678b46 100644
--- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
+++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
@@ -251,6 +251,7 @@ public abstract class Interpreter {
private String className;
private boolean defaultInterpreter;
private Map<String, InterpreterProperty> properties;
+ private Map<String, Object> editor;
private String path;
public RegisteredInterpreter(String name, String group, String className,
@@ -266,6 +267,7 @@ public abstract class Interpreter {
this.className = className;
this.defaultInterpreter = defaultInterpreter;
this.properties = properties;
+ this.editor = new HashMap<>();
}
public String getName() {
@@ -292,6 +294,10 @@ public abstract class Interpreter {
return properties;
}
+ public Map<String, Object> getEditor() {
+ return editor;
+ }
+
public void setPath(String path) {
this.path = path;
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
----------------------------------------------------------------------
diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
index 8b6329a..3e54e05 100644
--- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
+++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
@@ -250,6 +250,9 @@ public class NotebookServer extends WebSocketServlet implements
case SAVE_INTERPRETER_BINDINGS:
saveInterpreterBindings(conn, messagereceived);
break;
+ case EDITOR_SETTING:
+ getEditorSetting(conn, messagereceived);
+ break;
default:
break;
}
@@ -1457,7 +1460,7 @@ public class NotebookServer extends WebSocketServlet implements
}
/**
- * This callback is for praragraph that runs on RemoteInterpreterProcess
+ * This callback is for paragraph that runs on RemoteInterpreterProcess
* @param paragraph
* @param out
* @param output
@@ -1569,11 +1572,23 @@ public class NotebookServer extends WebSocketServlet implements
if (id.equals(interpreterGroupId)) {
broadcast(
note.getId(),
- new Message(OP.ANGULAR_OBJECT_REMOVE).put("name", name).put(
- "noteId", noteId).put("paragraphId", paragraphId));
+ new Message(OP.ANGULAR_OBJECT_REMOVE)
+ .put("name", name)
+ .put("noteId", noteId)
+ .put("paragraphId", paragraphId));
}
}
}
}
+
+ private void getEditorSetting(NotebookSocket conn, Message fromMessage)
+ throws IOException {
+ String replName = (String) fromMessage.get("magic");
+ String noteId = getOpenNoteId(conn);
+ Message resp = new Message(OP.EDITOR_SETTING);
+ resp.put("editor", notebook().getInterpreterFactory().getEditorSetting(noteId, replName));
+ conn.send(serializeMessage(resp));
+ return;
+ }
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/bower.json
----------------------------------------------------------------------
diff --git a/zeppelin-web/bower.json b/zeppelin-web/bower.json
index 5ec18a7..aeabd79 100644
--- a/zeppelin-web/bower.json
+++ b/zeppelin-web/bower.json
@@ -24,7 +24,7 @@
"angular-elastic": "~2.4.2",
"angular-elastic-input": "~2.2.0",
"angular-xeditable": "0.1.12",
- "highlightjs": "^9.2.0",
+ "highlightjs": "^9.4.0",
"lodash": "~3.9.3",
"angular-filter": "~0.5.4",
"ngtoast": "~2.0.0",
@@ -62,7 +62,7 @@
"highlight.pack.js",
"styles/github.css"
],
- "version": "8.4.0",
+ "version": "9.4.0",
"name": "highlightjs"
}
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
index bd3b6b3..8c46810 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
@@ -15,13 +15,14 @@
angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $rootScope, $route, $window,
$routeParams, $location, $timeout, $compile,
- $http, websocketMsgSrv, baseUrlSrv, ngToast,
+ $http, $q, websocketMsgSrv, baseUrlSrv, ngToast,
saveAsService, esriLoader) {
var ANGULAR_FUNCTION_OBJECT_NAME_PREFIX = '_Z_ANGULAR_FUNC_';
$scope.parentNote = null;
$scope.paragraph = null;
$scope.originalText = '';
$scope.editor = null;
+ $scope.magic = null;
var paragraphScope = $rootScope.$new(true, $rootScope);
@@ -78,15 +79,6 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
var angularObjectRegistry = {};
- var editorModes = {
- 'ace/mode/python': /^%(\w*\.)?(pyspark|python)\s*$/,
- 'ace/mode/scala': /^%(\w*\.)?spark\s*$/,
- 'ace/mode/r': /^%(\w*\.)?(r|sparkr|knitr)\s*$/,
- 'ace/mode/sql': /^%(\w*\.)?\wql/,
- 'ace/mode/markdown': /^%md/,
- 'ace/mode/sh': /^%sh/
- };
-
// Controller init
$scope.init = function(newParagraph, note) {
$scope.paragraph = newParagraph;
@@ -538,7 +530,7 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
$scope.aceChanged = function() {
$scope.dirtyText = $scope.editor.getSession().getValue();
$scope.startSaveTimer();
- $scope.setParagraphMode($scope.editor.getSession(), $scope.dirtyText, $scope.editor.getCursorPosition());
+ setParagraphMode($scope.editor.getSession(), $scope.dirtyText, $scope.editor.getCursorPosition());
};
$scope.aceLoaded = function(_editor) {
@@ -576,37 +568,11 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
// not applying emacs key binding while the binding override Ctrl-v. default behavior of paste text on windows.
}
- $scope.setParagraphMode = function(session, paragraphText, pos) {
- // Evaluate the mode only if the first 30 characters of the paragraph have been modified or the the position is undefined.
- if ((typeof pos === 'undefined') || (pos.row === 0 && pos.column < 30)) {
- // If paragraph loading, use config value if exists
- if ((typeof pos === 'undefined') && $scope.paragraph.config.editorMode) {
- session.setMode($scope.paragraph.config.editorMode);
- } else {
- // Defaults to spark mode
- var newMode = 'ace/mode/scala';
- // Test first against current mode
- var oldMode = session.getMode().$id;
- if (!editorModes[oldMode] || !editorModes[oldMode].test(paragraphText)) {
- for (var key in editorModes) {
- if (key !== oldMode) {
- if (editorModes[key].test(paragraphText)) {
- $scope.paragraph.config.editorMode = key;
- session.setMode(key);
- return true;
- }
- }
- }
- $scope.paragraph.config.editorMode = newMode;
- session.setMode(newMode);
- }
- }
- }
- };
-
var remoteCompleter = {
getCompletions: function(editor, session, pos, prefix, callback) {
- if (!$scope.editor.isFocused()) { return;}
+ if (!$scope.editor.isFocused()) {
+ return;
+ }
pos = session.getTextRange(new Range(0, 0, pos.row, pos.column)).length;
var buf = session.getValue();
@@ -662,7 +628,7 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
autoAdjustEditorHeight(_editor.container.id);
});
- $scope.setParagraphMode($scope.editor.getSession(), $scope.editor.getSession().getValue());
+ setParagraphMode($scope.editor.getSession(), $scope.editor.getSession().getValue());
// autocomplete on '.'
/*
@@ -737,6 +703,53 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
}
};
+ var getAndSetEditorSetting = function(session, interpreterName) {
+ var deferred = $q.defer();
+ websocketMsgSrv.getEditorSetting(interpreterName);
+ $timeout(
+ $scope.$on('editorSetting', function(event, data) {
+ deferred.resolve(data);
+ }
+ ), 1000);
+
+ deferred.promise.then(function(editorSetting) {
+ if (!_.isEmpty(editorSetting.editor)) {
+ var mode = 'ace/mode/' + editorSetting.editor.language;
+ $scope.paragraph.config.editorMode = mode;
+ session.setMode(mode);
+ }
+ });
+ };
+
+ var setParagraphMode = function(session, paragraphText, pos) {
+ // Evaluate the mode only if the the position is undefined
+ // or the first 30 characters of the paragraph have been modified
+ // or cursor position is at beginning of second line.(in case user hit enter after typing %magic)
+ if ((typeof pos === 'undefined') || (pos.row === 0 && pos.column < 30) || (pos.row === 1 && pos.column === 0)) {
+ // If paragraph loading, use config value if exists
+ if ((typeof pos === 'undefined') && $scope.paragraph.config.editorMode) {
+ session.setMode($scope.paragraph.config.editorMode);
+ } else {
+ var magic;
+ // set editor mode to default interpreter syntax if paragraph text doesn't start with '%'
+ // TODO(mina): dig into the cause what makes interpreterBindings has no element
+ if (!paragraphText.startsWith('%') && ((typeof pos !== 'undefined') && pos.row === 0 && pos.column === 1) ||
+ (typeof pos === 'undefined') && $scope.$parent.interpreterBindings.length !== 0) {
+ magic = $scope.$parent.interpreterBindings[0].name;
+ getAndSetEditorSetting(session, magic);
+ } else {
+ var replNameRegexp = /%(.+?)\s/g;
+ var match = replNameRegexp.exec(paragraphText);
+ if (match && $scope.magic !== match[1]) {
+ magic = match[1].trim();
+ $scope.magic = magic;
+ getAndSetEditorSetting(session, magic);
+ }
+ }
+ }
+ }
+ };
+
var autoAdjustEditorHeight = function(id) {
var editor = $scope.editor;
var height = editor.getSession().getScreenLength() * editor.renderer.lineHeight +
@@ -2259,7 +2272,6 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
var noteId = $route.current.pathParams.noteId;
$http.get(baseUrlSrv.getRestApiBase() + '/helium/suggest/' + noteId + '/' + $scope.paragraph.id)
.success(function(data, status, headers, config) {
- console.log('Suggested apps %o', data);
$scope.suggestion = data.body;
})
.error(function(err, status, headers, config) {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js b/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
index e4f45db..36e90b4 100644
--- a/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
+++ b/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
@@ -76,8 +76,8 @@ angular.module('zeppelinWebApp').factory('websocketEvents',
action: function(dialog) {
dialog.close();
angular.element('#loginModal').modal({
- show: 'true'
- });
+ show: 'true'
+ });
}
}, {
label: 'Cancel',
@@ -97,6 +97,8 @@ angular.module('zeppelinWebApp').factory('websocketEvents',
$rootScope.$broadcast('updateProgress', data);
} else if (op === 'COMPLETION_LIST') {
$rootScope.$broadcast('completionList', data);
+ } else if (op === 'EDITOR_SETTING') {
+ $rootScope.$broadcast('editorSetting', data);
} else if (op === 'ANGULAR_OBJECT_UPDATE') {
$rootScope.$broadcast('angularObjectUpdate', data);
} else if (op === 'ANGULAR_OBJECT_REMOVE') {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js b/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
index 3dcdc3e..3b27bce 100644
--- a/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
+++ b/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
@@ -180,6 +180,15 @@ angular.module('zeppelinWebApp').service('websocketMsgSrv', function($rootScope,
});
},
+ getEditorSetting: function(replName) {
+ websocketEvents.sendNewEvent({
+ op: 'EDITOR_SETTING',
+ data: {
+ magic: replName
+ }
+ });
+ },
+
isConnected: function() {
return websocketEvents.isConnected();
},
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java
index 47ff946..40dcc30 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java
@@ -49,6 +49,8 @@ import java.util.Properties;
import java.util.Set;
import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.reflect.TypeToken;
@@ -223,7 +225,8 @@ public class InterpreterFactory implements InterpreterGroupFactory {
InterpreterInfo interpreterInfo;
for (RegisteredInterpreter r : Interpreter.registeredInterpreters.values()) {
interpreterInfo =
- new InterpreterInfo(r.getClassName(), r.getName(), r.isDefaultInterpreter());
+ new InterpreterInfo(r.getClassName(), r.getName(), r.isDefaultInterpreter(),
+ r.getEditor());
add(r.getGroup(), interpreterInfo, convertInterpreterProperties(r.getProperties()),
r.getPath());
}
@@ -335,7 +338,7 @@ public class InterpreterFactory implements InterpreterGroupFactory {
for (RegisteredInterpreter registeredInterpreter : registeredInterpreters) {
InterpreterInfo interpreterInfo =
new InterpreterInfo(registeredInterpreter.getClassName(), registeredInterpreter.getName(),
- registeredInterpreter.isDefaultInterpreter());
+ registeredInterpreter.isDefaultInterpreter(), registeredInterpreter.getEditor());
Properties properties = new Properties();
Map<String, InterpreterProperty> p = registeredInterpreter.getProperties();
@@ -370,10 +373,11 @@ public class InterpreterFactory implements InterpreterGroupFactory {
fis.close();
String json = sb.toString();
- InterpreterInfoSaving info = gson.fromJson(json, InterpreterInfoSaving.class);
+ InterpreterInfoSaving infoSaving = gson.fromJson(json, InterpreterInfoSaving.class);
- for (String k : info.interpreterSettings.keySet()) {
- InterpreterSetting setting = info.interpreterSettings.get(k);
+ for (String k : infoSaving.interpreterSettings.keySet()) {
+ InterpreterSetting setting = infoSaving.interpreterSettings.get(k);
+ List<InterpreterInfo> infos = setting.getInterpreterInfos();
// Always use separate interpreter process
// While we decided to turn this feature on always (without providing
@@ -391,15 +395,23 @@ public class InterpreterFactory implements InterpreterGroupFactory {
depClassPath = interpreterSettingObject.getPath();
setting.setPath(depClassPath);
+ for (InterpreterInfo info : infos) {
+ if (info.getEditor() == null) {
+ Map<String, Object> editor = getEditorFromSettingByClassName(interpreterSettingObject,
+ info.getClassName());
+ info.setEditor(editor);
+ }
+ }
+
setting.setInterpreterGroupFactory(this);
loadInterpreterDependencies(setting);
interpreterSettings.put(k, setting);
}
- this.interpreterBindings = info.interpreterBindings;
+ this.interpreterBindings = infoSaving.interpreterBindings;
- if (info.interpreterRepositories != null) {
- for (RemoteRepository repo : info.interpreterRepositories) {
+ if (infoSaving.interpreterRepositories != null) {
+ for (RemoteRepository repo : infoSaving.interpreterRepositories) {
if (!depResolver.getRepos().contains(repo)) {
this.interpreterRepositories.add(repo);
}
@@ -407,6 +419,17 @@ public class InterpreterFactory implements InterpreterGroupFactory {
}
}
+ public Map<String, Object> getEditorFromSettingByClassName(InterpreterSetting intpSetting,
+ String className) {
+ List<InterpreterInfo> intpInfos = intpSetting.getInterpreterInfos();
+ for (InterpreterInfo intpInfo : intpInfos) {
+ if (className.equals(intpInfo.getClassName())) {
+ return intpInfo.getEditor();
+ }
+ }
+ return ImmutableMap.of("language", (Object) "text");
+ }
+
private void loadInterpreterDependencies(final InterpreterSetting setting) {
setting.setStatus(InterpreterSetting.Status.DOWNLOADING_DEPENDENCIES);
@@ -1261,6 +1284,35 @@ public class InterpreterFactory implements InterpreterGroupFactory {
this.env = env;
}
+ public Map<String, Object> getEditorSetting(String noteId, String replName) {
+ Interpreter intp = getInterpreter(noteId, replName);
+ Map<String, Object> editor = Maps.newHashMap(
+ ImmutableMap.<String, Object>builder()
+ .put("language", "text").build());
+ String defaultSettingName = getDefaultInterpreterSetting(noteId).getName();
+ String group = StringUtils.EMPTY;
+ try {
+ List<InterpreterSetting> intpSettings = getInterpreterSettings(noteId);
+ for (InterpreterSetting intpSetting : intpSettings) {
+ String[] replNameSplit = replName.split("\\.");
+ if (replNameSplit.length == 2) {
+ group = replNameSplit[0];
+ }
+ // when replName is 'name' of interpreter
+ if (defaultSettingName.equals(intpSetting.getName())) {
+ editor = getEditorFromSettingByClassName(intpSetting, intp.getClassName());
+ }
+ // when replName is 'alias name' of interpreter or 'group' of interpreter
+ if (replName.equals(intpSetting.getName()) || group.equals(intpSetting.getName())) {
+ editor = getEditorFromSettingByClassName(intpSetting, intp.getClassName());
+ break;
+ }
+ }
+ } catch (NullPointerException e) {
+ logger.warn("Couldn't get interpreter editor language");
+ }
+ return editor;
+ }
private Interpreter getDevInterpreter() {
if (devInterpreter == null) {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java
index c104b9d..e9088e9 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java
@@ -19,6 +19,8 @@ package org.apache.zeppelin.interpreter;
import com.google.gson.annotations.SerializedName;
+import java.util.Map;
+
/**
* Information of interpreters in this interpreter setting.
* this will be serialized for conf/interpreter.json and REST api response.
@@ -27,11 +29,14 @@ public class InterpreterInfo {
private String name;
@SerializedName("class") private String className;
private boolean defaultInterpreter = false;
+ private Map<String, Object> editor;
- InterpreterInfo(String className, String name, boolean defaultInterpreter) {
+ InterpreterInfo(String className, String name, boolean defaultInterpreter,
+ Map<String, Object> editor) {
this.className = className;
this.name = name;
this.defaultInterpreter = defaultInterpreter;
+ this.editor = editor;
}
public String getName() {
@@ -50,6 +55,14 @@ public class InterpreterInfo {
return defaultInterpreter;
}
+ public Map<String, Object> getEditor() {
+ return editor;
+ }
+
+ public void setEditor(Map<String, Object> editor) {
+ this.editor = editor;
+ }
+
@Override
public boolean equals(Object obj) {
if (!(obj instanceof InterpreterInfo)) {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java
index 9175e30..b314e62 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java
@@ -21,43 +21,43 @@ import java.util.HashMap;
import java.util.Map;
/**
- * Zeppelin websocker massage template class.
+ * Zeppelin websocket massage template class.
*/
public class Message {
/**
* Representation of event type.
*/
public static enum OP {
- GET_HOME_NOTE, // [c-s] load note for home screen
+ GET_HOME_NOTE, // [c-s] load note for home screen
- GET_NOTE, // [c-s] client load note
- // @param id note id
+ GET_NOTE, // [c-s] client load note
+ // @param id note id
- NOTE, // [s-c] note info
- // @param note serlialized Note object
+ NOTE, // [s-c] note info
+ // @param note serlialized Note object
- PARAGRAPH, // [s-c] paragraph info
- // @param paragraph serialized paragraph object
+ PARAGRAPH, // [s-c] paragraph info
+ // @param paragraph serialized paragraph object
- PROGRESS, // [s-c] progress update
- // @param id paragraph id
- // @param progress percentage progress
-
- NEW_NOTE, // [c-s] create new notebook
- DEL_NOTE, // [c-s] delete notebook
- // @param id note id
- CLONE_NOTE, // [c-s] clone new notebook
- // @param id id of note to clone
- // @param name name fpor the cloned note
- IMPORT_NOTE, // [c-s] import notebook
- // @param object notebook
+ PROGRESS, // [s-c] progress update
+ // @param id paragraph id
+ // @param progress percentage progress
+
+ NEW_NOTE, // [c-s] create new notebook
+ DEL_NOTE, // [c-s] delete notebook
+ // @param id note id
+ CLONE_NOTE, // [c-s] clone new notebook
+ // @param id id of note to clone
+ // @param name name fpor the cloned note
+ IMPORT_NOTE, // [c-s] import notebook
+ // @param object notebook
NOTE_UPDATE,
- RUN_PARAGRAPH, // [c-s] run paragraph
- // @param id paragraph id
- // @param paragraph paragraph content.ie. script
- // @param config paragraph config
- // @param params paragraph params
+ RUN_PARAGRAPH, // [c-s] run paragraph
+ // @param id paragraph id
+ // @param paragraph paragraph content.ie. script
+ // @param config paragraph config
+ // @param params paragraph params
COMMIT_PARAGRAPH, // [c-s] commit paragraph
// @param id paragraph id
@@ -69,72 +69,77 @@ public class Message {
CANCEL_PARAGRAPH, // [c-s] cancel paragraph run
// @param id paragraph id
- MOVE_PARAGRAPH, // [c-s] move paragraph order
- // @param id paragraph id
- // @param index index the paragraph want to go
+ MOVE_PARAGRAPH, // [c-s] move paragraph order
+ // @param id paragraph id
+ // @param index index the paragraph want to go
INSERT_PARAGRAPH, // [c-s] create new paragraph below current paragraph
// @param target index
- COMPLETION, // [c-s] ask completion candidates
- // @param id
- // @param buf current code
- // @param cursor cursor position in code
+ EDITOR_SETTING, // [c-s] ask paragraph editor setting
+ // @param magic magic keyword written in paragraph
+ // ex) spark.spark or spark
+
+ COMPLETION, // [c-s] ask completion candidates
+ // @param id
+ // @param buf current code
+ // @param cursor cursor position in code
- COMPLETION_LIST, // [s-c] send back completion candidates list
- // @param id
- // @param completions list of string
+ COMPLETION_LIST, // [s-c] send back completion candidates list
+ // @param id
+ // @param completions list of string
- LIST_NOTES, // [c-s] ask list of note
- RELOAD_NOTES_FROM_REPO, // [c-s] reload notes from repo
+ LIST_NOTES, // [c-s] ask list of note
+ RELOAD_NOTES_FROM_REPO, // [c-s] reload notes from repo
- NOTES_INFO, // [s-c] list of note infos
- // @param notes serialized List<NoteInfo> object
+ NOTES_INFO, // [s-c] list of note infos
+ // @param notes serialized List<NoteInfo> object
PARAGRAPH_REMOVE,
PARAGRAPH_CLEAR_OUTPUT,
- PARAGRAPH_APPEND_OUTPUT, // [s-c] append output
- PARAGRAPH_UPDATE_OUTPUT, // [s-c] update (replace) output
+ PARAGRAPH_APPEND_OUTPUT, // [s-c] append output
+ PARAGRAPH_UPDATE_OUTPUT, // [s-c] update (replace) output
PING,
AUTH_INFO,
- ANGULAR_OBJECT_UPDATE, // [s-c] add/update angular object
- ANGULAR_OBJECT_REMOVE, // [s-c] add angular object del
+ ANGULAR_OBJECT_UPDATE, // [s-c] add/update angular object
+ ANGULAR_OBJECT_REMOVE, // [s-c] add angular object del
- ANGULAR_OBJECT_UPDATED, // [c-s] angular object value updated,
+ ANGULAR_OBJECT_UPDATED, // [c-s] angular object value updated,
- ANGULAR_OBJECT_CLIENT_BIND, // [c-s] angular object updated from AngularJS z object
+ ANGULAR_OBJECT_CLIENT_BIND, // [c-s] angular object updated from AngularJS z object
- ANGULAR_OBJECT_CLIENT_UNBIND, // [c-s] angular object unbind from AngularJS z object
+ ANGULAR_OBJECT_CLIENT_UNBIND, // [c-s] angular object unbind from AngularJS z object
- LIST_CONFIGURATIONS, // [c-s] ask all key/value pairs of configurations
- CONFIGURATIONS_INFO, // [s-c] all key/value pairs of configurations
- // @param settings serialized Map<String, String> object
+ LIST_CONFIGURATIONS, // [c-s] ask all key/value pairs of configurations
+ CONFIGURATIONS_INFO, // [s-c] all key/value pairs of configurations
+ // @param settings serialized Map<String, String> object
- CHECKPOINT_NOTEBOOK, // [c-s] checkpoint notebook to storage repository
- // @param noteId
- // @param checkpointName
+ CHECKPOINT_NOTEBOOK, // [c-s] checkpoint notebook to storage repository
+ // @param noteId
+ // @param checkpointName
- LIST_REVISION_HISTORY, // [c-s] list revision history of the notebook
- // @param noteId
- NOTE_REVISION, // [c-s] get certain revision of note
- // @param noteId
- // @param revisionId
+ LIST_REVISION_HISTORY, // [c-s] list revision history of the notebook
+ // @param noteId
+ NOTE_REVISION, // [c-s] get certain revision of note
+ // @param noteId
+ // @param revisionId
- APP_APPEND_OUTPUT, // [s-c] append output
- APP_UPDATE_OUTPUT, // [s-c] update (replace) output
- APP_LOAD, // [s-c] on app load
- APP_STATUS_CHANGE, // [s-c] on app status change
+ APP_APPEND_OUTPUT, // [s-c] append output
+ APP_UPDATE_OUTPUT, // [s-c] update (replace) output
+ APP_LOAD, // [s-c] on app load
+ APP_STATUS_CHANGE, // [s-c] on app status change
- LIST_NOTEBOOK_JOBS, // [c-s] get notebook job management infomations
- LIST_UPDATE_NOTEBOOK_JOBS, // [s-c] get job management informations
+ LIST_NOTEBOOK_JOBS, // [c-s] get notebook job management infomations
+ LIST_UPDATE_NOTEBOOK_JOBS, // [c-s] get job management informations for until unixtime
UNSUBSCRIBE_UPDATE_NOTEBOOK_JOBS, // [c-s] unsubscribe job information for job management
- GET_INTERPRETER_BINDINGS, // [c-s] get interpreter bindings
- // @param noteID
- SAVE_INTERPRETER_BINDINGS, // [c-s] save interpreter bindings
- // @param noteID
- // @param selectedSettingIds
- INTERPRETER_BINDINGS // [s-c] interpreter bindings
+ // @param unixTime
+ GET_INTERPRETER_BINDINGS, // [c-s] get interpreter bindings
+ // @param noteID
+ SAVE_INTERPRETER_BINDINGS, // [c-s] save interpreter bindings
+ // @param noteID
+ // @param selectedSettingIds
+ INTERPRETER_BINDINGS // [s-c] interpreter bindings
}
public OP op;
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java
index 5cdda05..09031a5 100644
--- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java
+++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java
@@ -34,12 +34,22 @@ import org.apache.zeppelin.dep.DependencyResolver;
import org.apache.zeppelin.interpreter.mock.MockInterpreter1;
import org.apache.zeppelin.interpreter.mock.MockInterpreter2;
import org.apache.zeppelin.interpreter.remote.RemoteInterpreter;
+import org.apache.zeppelin.notebook.JobListenerFactory;
+import org.apache.zeppelin.notebook.Note;
+import org.apache.zeppelin.notebook.Notebook;
+import org.apache.zeppelin.notebook.repo.NotebookRepo;
+import org.apache.zeppelin.notebook.repo.VFSNotebookRepo;
+import org.apache.zeppelin.scheduler.SchedulerFactory;
+import org.apache.zeppelin.search.SearchService;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
+import org.quartz.SchedulerException;
import org.sonatype.aether.RepositoryException;
import static org.junit.Assert.*;
+import static org.mockito.Mockito.mock;
+import org.mockito.Mock;
public class InterpreterFactoryTest {
@@ -47,13 +57,19 @@ public class InterpreterFactoryTest {
private File tmpDir;
private ZeppelinConfiguration conf;
private InterpreterContext context;
+ private Notebook notebook;
+ private NotebookRepo notebookRepo;
private DependencyResolver depResolver;
+ private SchedulerFactory schedulerFactory;
+ @Mock
+ private JobListenerFactory jobListenerFactory;
@Before
public void setUp() throws Exception {
tmpDir = new File(System.getProperty("java.io.tmpdir")+"/ZeppelinLTest_"+System.currentTimeMillis());
tmpDir.mkdirs();
new File(tmpDir, "conf").mkdirs();
+ FileUtils.copyDirectory(new File("src/test/resources/interpreter"), new File(tmpDir, "interpreter"));
Map<String, InterpreterProperty> propertiesMockInterpreter1 = new HashMap<String, InterpreterProperty>();
propertiesMockInterpreter1.put("PROPERTY_1", new InterpreterProperty("PROPERTY_1", "", "VALUE_1", "desc"));
@@ -62,11 +78,22 @@ public class InterpreterFactoryTest {
MockInterpreter2.register("mock2", "org.apache.zeppelin.interpreter.mock.MockInterpreter2");
System.setProperty(ConfVars.ZEPPELIN_HOME.getVarName(), tmpDir.getAbsolutePath());
- System.setProperty(ConfVars.ZEPPELIN_INTERPRETERS.getVarName(), "org.apache.zeppelin.interpreter.mock.MockInterpreter1,org.apache.zeppelin.interpreter.mock.MockInterpreter2");
+ System.setProperty(ConfVars.ZEPPELIN_INTERPRETERS.getVarName(),
+ "org.apache.zeppelin.interpreter.mock.MockInterpreter1," +
+ "org.apache.zeppelin.interpreter.mock.MockInterpreter2," +
+ "org.apache.zeppelin.interpreter.mock.MockInterpreter11");
+ System.setProperty(ConfVars.ZEPPELIN_INTERPRETER_GROUP_ORDER.getVarName(),
+ "mock1,mock2,mock11,dev");
conf = new ZeppelinConfiguration();
+ schedulerFactory = new SchedulerFactory();
depResolver = new DependencyResolver(tmpDir.getAbsolutePath() + "/local-repo");
factory = new InterpreterFactory(conf, new InterpreterOption(false), null, null, null, depResolver);
context = new InterpreterContext("note", "id", "title", "text", null, null, null, null, null, null, null);
+
+ SearchService search = mock(SearchService.class);
+ notebookRepo = new VFSNotebookRepo(conf);
+ notebook = new Notebook(conf, notebookRepo, schedulerFactory, factory, jobListenerFactory, search,
+ null, null);
}
@After
@@ -128,7 +155,7 @@ public class InterpreterFactoryTest {
public void testFactoryDefaultList() throws IOException, RepositoryException {
// get default settings
List<String> all = factory.getDefaultInterpreterSettingList();
- assertTrue(factory.getRegisteredInterpreterList().size() >= all.size());
+ assertTrue(factory.get().size() >= all.size());
}
@Test
@@ -166,8 +193,8 @@ public class InterpreterFactoryTest {
@Test
public void testInterpreterAliases() throws IOException, RepositoryException {
factory = new InterpreterFactory(conf, null, null, null, depResolver);
- final InterpreterInfo info1 = new InterpreterInfo("className1", "name1", true);
- final InterpreterInfo info2 = new InterpreterInfo("className2", "name1", true);
+ final InterpreterInfo info1 = new InterpreterInfo("className1", "name1", true, null);
+ final InterpreterInfo info2 = new InterpreterInfo("className2", "name1", true, null);
factory.add("group1", new ArrayList<InterpreterInfo>(){{
add(info1);
}}, new ArrayList<Dependency>(), new InterpreterOption(true), new Properties(), "/path1");
@@ -196,4 +223,29 @@ public class InterpreterFactoryTest {
assertEquals("'.' is invalid for InterpreterSetting name.", e.getMessage());
}
}
+
+
+ @Test
+ public void getEditorSetting() throws IOException, RepositoryException, SchedulerException {
+ List<String> intpIds = new ArrayList<>();
+ for(InterpreterSetting intpSetting: factory.get()) {
+ if (intpSetting.getName().startsWith("mock1")) {
+ intpIds.add(intpSetting.getId());
+ }
+ }
+ Note note = notebook.createNote(intpIds, null);
+
+ // get editor setting from interpreter-setting.json
+ Map<String, Object> editor = factory.getEditorSetting(note.getId(), "mock11");
+ assertEquals("java", editor.get("language"));
+
+ // when interpreter is not loaded via interpreter-setting.json
+ // or editor setting doesn't exit
+ editor = factory.getEditorSetting(note.getId(), "mock1");
+ assertEquals(null, editor.get("language"));
+
+ // when interpreter is not bound to note
+ editor = factory.getEditorSetting(note.getId(), "mock2");
+ assertEquals("text", editor.get("language"));
+ }
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java
index 98e301a..a44bfad 100644
--- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java
+++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java
@@ -19,7 +19,6 @@ package org.apache.zeppelin.notebook;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.interpreter.InterpreterFactory;
-import org.apache.zeppelin.interpreter.InterpreterSetting;
import org.apache.zeppelin.notebook.repo.NotebookRepo;
import org.apache.zeppelin.scheduler.Scheduler;
import org.apache.zeppelin.search.SearchService;
@@ -28,7 +27,6 @@ import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
-import org.mockito.Mockito;
import org.mockito.runners.MockitoJUnitRunner;
import static org.junit.Assert.*;
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java
index 0ec8e7c..18d343c 100644
--- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java
+++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java
@@ -75,6 +75,7 @@ public class NotebookTest implements JobListenerFactory{
notebookDir = new File(tmpDir + "/notebook");
notebookDir.mkdirs();
+ System.setProperty(ConfVars.ZEPPELIN_CONF_DIR.getVarName(), tmpDir.toString() + "/conf");
System.setProperty(ConfVars.ZEPPELIN_HOME.getVarName(), tmpDir.getAbsolutePath());
System.setProperty(ConfVars.ZEPPELIN_NOTEBOOK_DIR.getVarName(), notebookDir.getAbsolutePath());
System.setProperty(ConfVars.ZEPPELIN_INTERPRETERS.getVarName(), "org.apache.zeppelin.interpreter.mock.MockInterpreter1,org.apache.zeppelin.interpreter.mock.MockInterpreter2");
@@ -95,8 +96,7 @@ public class NotebookTest implements JobListenerFactory{
credentials = new Credentials(conf.credentialsPersist(), conf.getCredentialsPath());
notebook = new Notebook(conf, notebookRepo, schedulerFactory, factory, this, search,
- notebookAuthorization, credentials);
-
+ notebookAuthorization, credentials);
}
@After
@@ -109,7 +109,7 @@ public class NotebookTest implements JobListenerFactory{
Note note = notebook.createNote(null);
factory.setInterpreters(note.getId(), factory.getDefaultInterpreterSettingList());
- // run with defatul repl
+ // run with default repl
Paragraph p1 = note.addParagraph();
Map config = p1.getConfig();
config.put("enabled", true);
@@ -232,7 +232,7 @@ public class NotebookTest implements JobListenerFactory{
p1.setText("hello world");
note.run(p1.getId());
- while(p1.isTerminated()==false || p1.getResult()==null) Thread.yield();
+ while(p1.isTerminated() == false || p1.getResult() == null) Thread.yield();
assertEquals("repl1: hello world", p1.getResult().message());
// clear paragraph output/result
@@ -267,7 +267,7 @@ public class NotebookTest implements JobListenerFactory{
note.runAll();
// wait for finish
- while(p3.isTerminated()==false) {
+ while(p3.isTerminated() == false) {
Thread.yield();
}
@@ -279,7 +279,7 @@ public class NotebookTest implements JobListenerFactory{
}
@Test
- public void testSchedule() throws InterruptedException, IOException{
+ public void testSchedule() throws InterruptedException, IOException {
// create a note and a paragraph
Note note = notebook.createNote(null);
factory.setInterpreters(note.getId(), factory.getDefaultInterpreterSettingList());
@@ -297,7 +297,7 @@ public class NotebookTest implements JobListenerFactory{
config.put("cron", "* * * * * ?");
note.setConfig(config);
notebook.refreshCron(note.getId());
- Thread.sleep(1*1000);
+ Thread.sleep(1 * 1000);
// remove cron scheduler.
config.put("cron", null);
@@ -306,7 +306,7 @@ public class NotebookTest implements JobListenerFactory{
Thread.sleep(1000);
dateFinished = p.getDateFinished();
assertNotNull(dateFinished);
- Thread.sleep(1*1000);
+ Thread.sleep(1 * 1000);
assertEquals(dateFinished, p.getDateFinished());
}
@@ -476,8 +476,8 @@ public class NotebookTest implements JobListenerFactory{
p2.setText("%mock2 world");
note.runAll();
- while(p1.isTerminated()==false || p1.getResult()==null) Thread.yield();
- while(p2.isTerminated()==false || p2.getResult()==null) Thread.yield();
+ while (p1.isTerminated() == false || p1.getResult() == null) Thread.yield();
+ while (p2.isTerminated() == false || p2.getResult() == null) Thread.yield();
assertEquals(2, ResourcePoolUtils.getAllResources().size());
@@ -604,26 +604,26 @@ public class NotebookTest implements JobListenerFactory{
new HashSet<String>(Arrays.asList("user1")));
assertEquals(notebookAuthorization.isOwner(note.getId(),
- new HashSet<String>(Arrays.asList("user2"))), false);
+ new HashSet<String>(Arrays.asList("user2"))), false);
assertEquals(notebookAuthorization.isOwner(note.getId(),
new HashSet<String>(Arrays.asList("user1"))), true);
assertEquals(notebookAuthorization.isReader(note.getId(),
- new HashSet<String>(Arrays.asList("user3"))), false);
+ new HashSet<String>(Arrays.asList("user3"))), false);
assertEquals(notebookAuthorization.isReader(note.getId(),
- new HashSet<String>(Arrays.asList("user2"))), true);
+ new HashSet<String>(Arrays.asList("user2"))), true);
assertEquals(notebookAuthorization.isWriter(note.getId(),
- new HashSet<String>(Arrays.asList("user2"))), false);
+ new HashSet<String>(Arrays.asList("user2"))), false);
assertEquals(notebookAuthorization.isWriter(note.getId(),
- new HashSet<String>(Arrays.asList("user1"))), true);
+ new HashSet<String>(Arrays.asList("user1"))), true);
// Test clearing of permssions
notebookAuthorization.setReaders(note.getId(), Sets.<String>newHashSet());
assertEquals(notebookAuthorization.isReader(note.getId(),
- new HashSet<String>(Arrays.asList("user2"))), true);
+ new HashSet<String>(Arrays.asList("user2"))), true);
assertEquals(notebookAuthorization.isReader(note.getId(),
- new HashSet<String>(Arrays.asList("user3"))), true);
+ new HashSet<String>(Arrays.asList("user3"))), true);
notebook.removeNote(note.getId(), null);
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json b/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json
new file mode 100644
index 0000000..65568ef
--- /dev/null
+++ b/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json
@@ -0,0 +1,12 @@
+[
+ {
+ "group": "mock11",
+ "name": "mock11",
+ "className": "org.apache.zeppelin.interpreter.mock.MockInterpreter11",
+ "properties": {
+ },
+ "editor": {
+ "language": "java"
+ }
+ }
+]