You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by zj...@apache.org on 2018/02/13 03:28:26 UTC
zeppelin git commit: [ZEPPELIN-3092] GitHub Integration
Repository: zeppelin
Updated Branches:
refs/heads/master d1293c6bc -> 2be8f3506
[ZEPPELIN-3092] GitHub Integration
### What is this PR for?
GitHub integration as a storage for notebooks.
### What type of PR is it?
Feature
### What is the Jira issue?
[ZEPPELIN-3092](https://issues.apache.org/jira/browse/ZEPPELIN-3092)
### How should this be tested?
1. Change the configuration in `zeppelin-site.xml` to enable GitHub integration (add GitHub url, username, access token and origin) as described in https://github.com/apache/zeppelin/compare/master...mohamagdy:zeppelin-3092-remote-github-integration?expand=1#diff-89104d48f0358450399a6f679bba9c4f
2. Start the Zeppelin server
3. Open an existing notebook or create a new notebook
4. Do some changes to the notebook, for example add a new paragraph
5. Click on the versioning button on the top menu to commit and save changes
6. Checkout the changes in the GitHub repository. The changes should be reflected
### Questions:
* **Does the licenses files need update?**
No
* **Is there breaking changes for older versions?**
No
* **Does this needs documentation?**
Yes. Documentation is updated as part of the pull request.
Author: Mohamed Magdy <mo...@fyber.com>
Author: Mohamed Magdy <mo...@smartfrog.com>
Author: Mohamed Magdy <en...@gmail.com>
Closes #2700 from mohamagdy/zeppelin-3092-remote-github-integration and squashes the following commits:
b445960 [Mohamed Magdy] [ZEPPELIN-3092] Optimize imports for `Notebook` class
afa5de1 [Mohamed Magdy] Merge branch 'master' of github.com:apache/zeppelin into zeppelin-3092-remote-github-integration
548c423 [Mohamed Magdy] [ZEPPELIN-3092] Add `zeppelin-site.xml` to `.gitignore`
e98d1b0 [Mohamed Magdy] [ZEPPELIN-3092] Remove `zeppelin-site.xml` from Zeppelin Server resources
7a02855 [Mohamed Magdy] [ZEPPELIN-3092] Add Apache Software Foundation header
9101e58 [Mohamed Magdy] [ZEPPELIN-3092] Replace `printStackTrace()` with error logging
db94d55 [Mohamed Magdy] [ZEPPELIN-3092] Remove loading notebook from repository when requested
af952a0 [Mohamed Magdy] [ZEPPELIN-3029] Change authentication to `anonymous` instead of `empty`
b5fbc1e [Mohamed Magdy] [ZEPPELIN-3092] Break long line to smaller ones
4d6cc76 [Mohamed Magdy] [ZEPPELIN-3092] Load notebook from repository when requested
d1d43eb [Mohamed Magdy] Merge branch 'zeppelin-3092-remote-github-integration' of github.com:mohamagdy/zeppelin into zeppelin-3092-remote-github-integration
579bd6f [Mohamed Magdy] [ZEPPELIN-3092] Load note from memory when reloading
2f1b8bc [Mohamed Magdy] [ZEPPELIN-3092] Load note from memory when reloading
d545e81 [Mohamed Magdy] [ZEPPELIN-3092] Remove duplicated dependency from `pom.xml`
fc13fa6 [Mohamed Magdy] Revert "[ZEPPELIN-3029] Increase Paragraph and Browser timeouts"
be2c278 [Mohamed Magdy] Revert "[ZEPPELIN-3092] Set browser timeout to 180 seconds"
f362dcb [Mohamed Magdy] [ZEPPELIN-3029] Use jGit version 4.5.4 instead of 4.3.1
8bd23d0 [Mohamed Magdy] [ZEPPELIN-3092] Set browser timeout to 180 seconds
30f2ab4 [Mohamed Magdy] [ZEPPELIN-3029] Increase Paragraph and Browser timeouts
13a0014 [Mohamed Magdy] [ZEPPELIN-3092] Disable GitHub configuration for Zeppelin server
14cb024 [Mohamed Magdy] [ZEPPELIN-3092] Fix notebook path for Git and GitHub tests
0e9db3f [Mohamed Magdy] [ZEPPELIN-3092] Remove test GitHub repository URL and access token
90de14c [Mohamed Magdy] Merge branch 'master' into zeppelin-3092-remote-github-integration
2c1cf74 [Mohamed Magdy] [ZEPPELIN-3029] Fix remote origin key name
6ba67ca [Mohamed Magdy] [ZEPPELIN-3092] Add Javadoc to `GitHubNotebookRepo` and fix line length to 100
264565b [Mohamed Magdy] [ZEPPELIN-3092] Fix line length to be 100
0174bbd [Mohamed Magdy] [ZEPPELIN-3092] Add documentation how to enabled `GitHubNotebookRepo`
81969e1 [Mohamed Magdy] [ZEPPELIN-3092] Add documentation for loading notebooks from repo
3009abd [Mohamed Magdy] [ZEPPELIN-3092] Reset `GitNotebookRepo` to `master`
6aa4ba7 [Mohamed Magdy] [ZEPPELIN-3092] Revert back `GitNotebookRepo` to `master`
b77a2d3 [Mohamed Magdy] [ZEPPELIN-3092] Fix identation in `pom.xml`
aadd9b5 [Mohamed Magdy] [ZEPPELIN-3092] Revert back ZeppelinServer changes
0dacbf1 [Mohamed Magdy] [ZEPPELIN-3092] Fix encoding in the documenation
2b093b2 [Mohamed Magdy] [ZEPPELIN-3092] Add documentation about GitHub integration
843e42a [Mohamed Magdy] [ZEPPELIN-3092] Cleanup GitHub repository tests
5236176 [Mohamed Magdy] [ZEPPELIN-3092] Move GitHub notebook repostiory to separte file
2dbf116 [Mohamed Magdy] [ZEPPELIN-3092] Add GitHub configuration to `zeppelin-site.xml` template
bb0afe2 [Mohamed Magdy] [ZEPPELIN-3092] Add GitHub remote to configurations
33ae24a [Mohamed Magdy] [ZEPPELIN-3092] Add remote Github repository synchronzing
32f6764 [Mohamed Magdy] [ZEPPELIN-3092] Fix GitNotebook test
eeb485a [Mohamed Magdy] [ZEPPELIN-3092] Add Github configuration reader
0bde310 [Mohamed Magdy] [ZEPPELIN-3092] Add `zeppelin-site.xml` to `zeppelin-server` resources
9467503 [Mohamed Magdy] [ZEPPELIN-3092] Add `zepplein-server/local-repo` to `.gitignore`
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/2be8f350
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/2be8f350
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/2be8f350
Branch: refs/heads/master
Commit: 2be8f350658076c33d9d905b9e9907aa3d3a8792
Parents: d1293c6
Author: Mohamed Magdy <mo...@fyber.com>
Authored: Wed Jan 24 10:11:15 2018 +0100
Committer: Jeff Zhang <zj...@apache.org>
Committed: Tue Feb 13 11:28:19 2018 +0800
----------------------------------------------------------------------
.gitignore | 3 +
conf/zeppelin-site.xml.template | 24 ++
.../contribution/how_to_contribute_code.md | 12 +
docs/setup/operation/configuration.md | 30 +-
docs/setup/storage/storage.md | 42 +++
.../zeppelin/conf/ZeppelinConfiguration.java | 22 +-
.../apache/zeppelin/socket/NotebookServer.java | 2 +-
.../src/test/resources/2A94M5J1Z/note.json | 376 +++++++++++++++++++
.../src/test/resources/2A94M5J2Z/note.json | 376 +++++++++++++++++++
.../org/apache/zeppelin/notebook/Notebook.java | 38 +-
.../notebook/repo/GitHubNotebookRepo.java | 126 +++++++
.../zeppelin/notebook/repo/GitNotebookRepo.java | 5 +-
.../notebook/repo/GitHubNotebookRepoTest.java | 207 ++++++++++
.../notebook/repo/GitNotebookRepoTest.java | 16 +-
14 files changed, 1243 insertions(+), 36 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/.gitignore
----------------------------------------------------------------------
diff --git a/.gitignore b/.gitignore
index 773edc8..4086a4b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -18,6 +18,9 @@ spark-1.*-bin-hadoop*
lens/lens-cli-hist.log
+# Zeppelin server
+zeppelin-server/local-repo
+zeppelin-server/src/main/resources/zeppelin-site.xml
# conf file
conf/zeppelin-env.sh
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/conf/zeppelin-site.xml.template
----------------------------------------------------------------------
diff --git a/conf/zeppelin-site.xml.template b/conf/zeppelin-site.xml.template
index 33aa8ac..9e9898b 100755
--- a/conf/zeppelin-site.xml.template
+++ b/conf/zeppelin-site.xml.template
@@ -499,5 +499,29 @@
</property>
-->
+<!-- GitHub configurations
+<property>
+ <name>zeppelin.notebook.git.remote.url</name>
+ <value></value>
+ <description>remote Git repository URL</description>
+</property>
+<property>
+ <name>zeppelin.notebook.git.remote.username</name>
+ <value>token</value>
+ <description>remote Git repository username</description>
+</property>
+
+<property>
+ <name>zeppelin.notebook.git.remote.access-token</name>
+ <value></value>
+ <description>remote Git repository password</description>
+</property>
+
+<property>
+ <name>zeppelin.notebook.git.remote.origin</name>
+ <value>origin</value>
+ <description>Git repository remote</description>
+</property>
+-->
</configuration>
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/docs/development/contribution/how_to_contribute_code.md
----------------------------------------------------------------------
diff --git a/docs/development/contribution/how_to_contribute_code.md b/docs/development/contribution/how_to_contribute_code.md
index b172aa1..92b69b5 100644
--- a/docs/development/contribution/how_to_contribute_code.md
+++ b/docs/development/contribution/how_to_contribute_code.md
@@ -89,11 +89,17 @@ For the further
### Run Zeppelin server in development mode
+#### Option 1 - Command Line
+
+1. Copy the `conf/zeppelin-site.xml.template` to `zeppelin-server/src/main/resources/zeppelin-site.xml` and change the configurations in this file if required
+2. Run the following command
```
cd zeppelin-server
HADOOP_HOME=YOUR_HADOOP_HOME JAVA_HOME=YOUR_JAVA_HOME mvn exec:java -Dexec.mainClass="org.apache.zeppelin.server.ZeppelinServer" -Dexec.args=""
```
+#### Option 2 - Daemon Script
+
> **Note:** Make sure you first run ```mvn clean install -DskipTests``` on your zeppelin root directory, otherwise your server build will fail to find the required dependencies in the local repro.
or use daemon script
@@ -104,6 +110,12 @@ bin/zeppelin-daemon start
Server will be run on [http://localhost:8080](http://localhost:8080).
+#### Option 3 - IDE
+
+1. Copy the `conf/zeppelin-site.xml.template` to `zeppelin-server/src/main/resources/zeppelin-site.xml` and change the configurations in this file if required
+2. `ZeppelinServer.java` Main class
+
+
### Generating Thrift Code
Some portions of the Zeppelin code are generated by [Thrift](http://thrift.apache.org). For most Zeppelin changes, you don't need to worry about this. But if you modify any of the Thrift IDL files (e.g. zeppelin-interpreter/src/main/thrift/*.thrift), then you also need to regenerate these files and submit their updated version as part of your patch.
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/docs/setup/operation/configuration.md
----------------------------------------------------------------------
diff --git a/docs/setup/operation/configuration.md b/docs/setup/operation/configuration.md
index 1f4c6a2..ed4e1f2 100644
--- a/docs/setup/operation/configuration.md
+++ b/docs/setup/operation/configuration.md
@@ -329,6 +329,30 @@ If both are defined, then the **environment variables** will take priority.
<td>false</td>
<td>Enable directory listings on server.</td>
</tr>
+ <tr>
+ <td><h6 class="properties">ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL</h6></td>
+ <td><h6 class="properties">zeppelin.notebook.git.remote.url</h6></td>
+ <td></td>
+ <td>GitHub's repository URL. It could be either the HTTP URL or the SSH URL. For example git@github.com:apache/zeppelin.git</td>
+ </tr>
+ <tr>
+ <td><h6 class="properties">ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME</h6></td>
+ <td><h6 class="properties">zeppelin.notebook.git.remote.username</h6></td>
+ <td>token</td>
+ <td>GitHub username. By default it is `token` to use GitHub's API</td>
+ </tr>
+ <tr>
+ <td><h6 class="properties">ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN</h6></td>
+ <td><h6 class="properties">zeppelin.notebook.git.remote.access-token</h6></td>
+ <td>token</td>
+ <td>GitHub access token to use GitHub's API. If username/password combination is used and not GitHub API, then this value is the password</td>
+ </tr>
+ <tr>
+ <td><h6 class="properties">ZEPPELIN_NOTEBOOK_GIT_REMOTE_ORIGIN</h6></td>
+ <td><h6 class="properties">zeppelin.notebook.git.remote.origin</h6></td>
+ <td>token</td>
+ <td>GitHub remote name. Default is `origin`</td>
+ </tr>
</table>
@@ -431,7 +455,7 @@ The following properties needs to be updated in the `zeppelin-site.xml` in order
### Storing user credentials
-In order to avoid having to re-enter credentials every time you restart/redeploy Zeppelin, you can store the user credentials. Zeppelin supports this via the ZEPPELIN_CREDENTIALS_PERSIST configuration.
+In order to avoid having to re-enter credentials every time you restart/redeploy Zeppelin, you can store the user credentials. Zeppelin supports this via the ZEPPELIN_CREDENTIALS_PERSIST configuration.
Please notice that passwords will be stored in *plain text* by default. To encrypt the passwords, use the ZEPPELIN_CREDENTIALS_ENCRYPT_KEY config variable. This will encrypt passwords using the AES-128 algorithm.
@@ -473,5 +497,9 @@ update your configuration with the obfuscated password :
</property>
```
+### Create GitHub Access Token
+
+When using GitHub to track notebooks, one can use GitHub's API for authentication. To create an access token, please use the following link https://github.com/settings/tokens.
+The value of the access token generated is set in the `zeppelin.notebook.git.remote.access-token` property.
**Note:** After updating these configurations, Zeppelin server needs to be restarted.
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/docs/setup/storage/storage.md
----------------------------------------------------------------------
diff --git a/docs/setup/storage/storage.md b/docs/setup/storage/storage.md
index f6b8b5c..f34fc2c 100644
--- a/docs/setup/storage/storage.md
+++ b/docs/setup/storage/storage.md
@@ -34,6 +34,7 @@ There are few notebook storage systems available for a use out of the box:
* storage using Amazon S3 service - `S3NotebookRepo`
* storage using Azure service - `AzureNotebookRepo`
* storage using MongoDB - `MongoNotebookRepo`
+ * storage using GitHub - `GitHubNotebookRepo`
Multiple storage systems can be used at the same time by providing a comma-separated list of the class-names in the configuration.
By default, only first two of them will be automatically kept in sync by Zeppelin.
@@ -361,3 +362,44 @@ export ZEPPELIN_NOTEBOOK_MONGO_AUTOIMPORT=true
#### Import your local notes automatically
By setting `ZEPPELIN_NOTEBOOK_MONGO_AUTOIMPORT` as `true` (default `false`), you can import your local notes automatically when Zeppelin daemon starts up. This feature is for easy migration from local file system storage to MongoDB storage. A note with ID already existing in the collection will not be imported.
+
+## Notebook Storage in GitHub
+
+To enable GitHub tracking, uncomment the following properties in `zeppelin-site.xml`
+
+```sh
+<property>
+ <name>zeppelin.notebook.git.remote.url</name>
+ <value></value>
+ <description>remote Git repository URL</description>
+</property>
+
+<property>
+ <name>zeppelin.notebook.git.remote.username</name>
+ <value>token</value>
+ <description>remote Git repository username</description>
+</property>
+
+<property>
+ <name>zeppelin.notebook.git.remote.access-token</name>
+ <value></value>
+ <description>remote Git repository password</description>
+</property>
+
+<property>
+ <name>zeppelin.notebook.git.remote.origin</name>
+ <value>origin</value>
+ <description>Git repository remote</description>
+</property>
+```
+
+And set the `zeppelin.notebook.storage` propery to `org.apache.zeppelin.notebook.repo.GitHubNotebookRepo`
+
+```sh
+<property>
+ <name>zeppelin.notebook.storage</name>
+ <value>org.apache.zeppelin.notebook.repo.GitHubNotebookRepo</value>
+</property>
+```
+
+The access token could be obtained by following the steps on this link https://github.com/settings/tokens.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
----------------------------------------------------------------------
diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
index 6bce468..f7b3d7b 100644
--- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
+++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
@@ -561,6 +561,22 @@ public class ZeppelinConfiguration extends XMLConfiguration {
return getString(ConfVars.ZEPPELIN_INTERPRETER_LIFECYCLE_MANAGER_CLASS);
}
+ public String getZeppelinNotebookGitURL() {
+ return getString(ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL);
+ }
+
+ public String getZeppelinNotebookGitUsername() {
+ return getString(ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME);
+ }
+
+ public String getZeppelinNotebookGitAccessToken() {
+ return getString(ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN);
+ }
+
+ public String getZeppelinNotebookGitRemoteOrigin() {
+ return getString(ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_ORIGIN);
+ }
+
public Map<String, String> dumpConfigurations(ZeppelinConfiguration conf,
ConfigurationKeyPredicate predicate) {
Map<String, String> configurations = new HashMap<>();
@@ -745,8 +761,12 @@ public class ZeppelinConfiguration extends XMLConfiguration {
ZEPPELIN_INTERPRETER_LIFECYCLE_MANAGER_TIMEOUT_THRESHOLD(
"zeppelin.interpreter.lifecyclemanager.timeout.threshold", 3600000L),
- ZEPPELIN_OWNER_ROLE("zeppelin.notebook.default.owner.username", "");
+ ZEPPELIN_OWNER_ROLE("zeppelin.notebook.default.owner.username", ""),
+ ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL("zeppelin.notebook.git.remote.url", ""),
+ ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME("zeppelin.notebook.git.remote.username", "token"),
+ ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN("zeppelin.notebook.git.remote.access-token", ""),
+ ZEPPELIN_NOTEBOOK_GIT_REMOTE_ORIGIN("zeppelin.notebook.git.remote.origin", "origin");
private String varName;
@SuppressWarnings("rawtypes")
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
----------------------------------------------------------------------
diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
index 56aa50a..20d5ba9 100644
--- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
+++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
@@ -824,8 +824,8 @@ public class NotebookServer extends WebSocketServlet
String user = fromMessage.principal;
Note note = notebook.getNote(noteId);
- if (note != null) {
+ if (note != null) {
if (!hasParagraphReaderPermission(conn, notebook, noteId,
userAndRoles, fromMessage.principal, "read")) {
return;
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-server/src/test/resources/2A94M5J1Z/note.json
----------------------------------------------------------------------
diff --git a/zeppelin-server/src/test/resources/2A94M5J1Z/note.json b/zeppelin-server/src/test/resources/2A94M5J1Z/note.json
new file mode 100644
index 0000000..6e8e06f
--- /dev/null
+++ b/zeppelin-server/src/test/resources/2A94M5J1Z/note.json
@@ -0,0 +1,376 @@
+{
+ "paragraphs": [
+ {
+ "text": "%md\n## Welcome to Zeppelin.\n##### This is a live tutorial, you can run the code yourself. (Shift-Enter to Run)",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:32:15 PM",
+ "config": {
+ "colWidth": 12.0,
+ "editorHide": true,
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false,
+ "keys": [],
+ "values": [],
+ "groups": [],
+ "scatter": {}
+ }
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "markdown",
+ "editOnDblClick": true
+ },
+ "editorMode": "ace/mode/markdown",
+ "tableHide": false
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "HTML",
+ "data": "\u003cdiv class\u003d\"markdown-body\"\u003e\n\u003ch2\u003eWelcome to Zeppelin.\u003c/h2\u003e\n\u003ch5\u003eThis is a live tutorial, you can run the code yourself. (Shift-Enter to Run)\u003c/h5\u003e\n\u003c/div\u003e"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423836981412_-1007008116",
+ "id": "20150213-231621_168813393",
+ "dateCreated": "Feb 13, 2015 11:16:21 PM",
+ "dateStarted": "Dec 17, 2016 3:32:15 PM",
+ "dateFinished": "Dec 17, 2016 3:32:18 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "title": "Load data into table",
+ "text": "import org.apache.commons.io.IOUtils\nimport java.net.URL\nimport java.nio.charset.Charset\n\n// Zeppelin creates and injects sc (SparkContext) and sqlContext (HiveContext or SqlContext)\n// So you don\u0027t need create them manually\n\n// load bank data\nval bankText \u003d sc.parallelize(\n IOUtils.toString(\n new URL(\"https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv\"),\n Charset.forName(\"utf8\")).split(\"\\n\"))\n\ncase class Bank(age: Integer, job: String, marital: String, education: String, balance: Integer)\n\nval bank \u003d bankText.map(s \u003d\u003e s.split(\";\")).filter(s \u003d\u003e s(0) !\u003d \"\\\"age\\\"\").map(\n s \u003d\u003e Bank(s(0).toInt, \n s(1).replaceAll(\"\\\"\", \"\"),\n s(2).replaceAll(\"\\\"\", \"\"),\n s(3).replaceAll(\"\\\"\", \"\"),\n s(5).replaceAll(\"\\\"\", \"\").toInt\n )\n).toDF()\nbank.registerTempTable(\"bank\")",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:30:09 PM",
+ "config": {
+ "colWidth": 12.0,
+ "title": true,
+ "enabled": true,
+ "editorMode": "ace/mode/scala",
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false
+ }
+ }
+ ],
+ "editorSetting": {
+ "language": "scala",
+ "editOnDblClick": false
+ }
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TEXT",
+ "data": "import org.apache.commons.io.IOUtils\nimport java.net.URL\nimport java.nio.charset.Charset\nbankText: org.apache.spark.rdd.RDD[String] \u003d ParallelCollectionRDD[36] at parallelize at \u003cconsole\u003e:43\ndefined class Bank\nbank: org.apache.spark.sql.DataFrame \u003d [age: int, job: string ... 3 more fields]\nwarning: there were 1 deprecation warning(s); re-run with -deprecation for details\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423500779206_-1502780787",
+ "id": "20150210-015259_1403135953",
+ "dateCreated": "Feb 10, 2015 1:52:59 AM",
+ "dateStarted": "Dec 17, 2016 3:30:09 PM",
+ "dateFinished": "Dec 17, 2016 3:30:58 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%sql \nselect age, count(1) value\nfrom bank \nwhere age \u003c 30 \ngroup by age \norder by age",
+ "user": "anonymous",
+ "dateUpdated": "Mar 17, 2017 12:18:02 PM",
+ "config": {
+ "colWidth": 4.0,
+ "results": [
+ {
+ "graph": {
+ "mode": "multiBarChart",
+ "height": 366.0,
+ "optionOpen": false
+ },
+ "helium": {}
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "sql",
+ "editOnDblClick": false
+ },
+ "editorMode": "ace/mode/sql"
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TABLE",
+ "data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t20\n24\t24\n25\t44\n26\t77\n27\t94\n28\t103\n29\t97\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423500782552_-1439281894",
+ "id": "20150210-015302_1492795503",
+ "dateCreated": "Feb 10, 2015 1:53:02 AM",
+ "dateStarted": "Dec 17, 2016 3:30:13 PM",
+ "dateFinished": "Dec 17, 2016 3:31:04 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%sql \nselect age, count(1) value \nfrom bank \nwhere age \u003c ${maxAge\u003d30} \ngroup by age \norder by age",
+ "user": "anonymous",
+ "dateUpdated": "Mar 17, 2017 12:17:39 PM",
+ "config": {
+ "colWidth": 4.0,
+ "results": [
+ {
+ "graph": {
+ "mode": "multiBarChart",
+ "height": 294.0,
+ "optionOpen": false
+ },
+ "helium": {}
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "sql",
+ "editOnDblClick": false
+ },
+ "editorMode": "ace/mode/sql"
+ },
+ "settings": {
+ "params": {
+ "maxAge": "35"
+ },
+ "forms": {
+ "maxAge": {
+ "name": "maxAge",
+ "defaultValue": "30",
+ "hidden": false
+ }
+ }
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TABLE",
+ "data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t20\n24\t24\n25\t44\n26\t77\n27\t94\n28\t103\n29\t97\n30\t150\n31\t199\n32\t224\n33\t186\n34\t231\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423720444030_-1424110477",
+ "id": "20150212-145404_867439529",
+ "dateCreated": "Feb 12, 2015 2:54:04 PM",
+ "dateStarted": "Dec 17, 2016 3:30:58 PM",
+ "dateFinished": "Dec 17, 2016 3:31:07 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%sql \nselect age, count(1) value \nfrom bank \nwhere marital\u003d\"${marital\u003dsingle,single|divorced|married}\" \ngroup by age \norder by age",
+ "user": "anonymous",
+ "dateUpdated": "Mar 17, 2017 12:18:18 PM",
+ "config": {
+ "colWidth": 4.0,
+ "results": [
+ {
+ "graph": {
+ "mode": "stackedAreaChart",
+ "height": 280.0,
+ "optionOpen": false
+ },
+ "helium": {}
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "sql",
+ "editOnDblClick": false
+ },
+ "editorMode": "ace/mode/sql"
+ },
+ "settings": {
+ "params": {
+ "marital": "single"
+ },
+ "forms": {
+ "marital": {
+ "name": "marital",
+ "defaultValue": "single",
+ "options": [
+ {
+ "value": "single"
+ },
+ {
+ "value": "divorced"
+ },
+ {
+ "value": "married"
+ }
+ ],
+ "hidden": false
+ }
+ }
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TABLE",
+ "data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t17\n24\t13\n25\t33\n26\t56\n27\t64\n28\t78\n29\t56\n30\t92\n31\t86\n32\t105\n33\t61\n34\t75\n35\t46\n36\t50\n37\t43\n38\t44\n39\t30\n40\t25\n41\t19\n42\t23\n43\t21\n44\t20\n45\t15\n46\t14\n47\t12\n48\t12\n49\t11\n50\t8\n51\t6\n52\t9\n53\t4\n55\t3\n56\t3\n57\t2\n58\t7\n59\t2\n60\t5\n66\t2\n69\t1\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423836262027_-210588283",
+ "id": "20150213-230422_1600658137",
+ "dateCreated": "Feb 13, 2015 11:04:22 PM",
+ "dateStarted": "Dec 17, 2016 3:31:05 PM",
+ "dateFinished": "Dec 17, 2016 3:31:09 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%md\n## Congratulations, it\u0027s done.\n##### You can create your own notebook in \u0027Notebook\u0027 menu. Good luck!",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:30:24 PM",
+ "config": {
+ "colWidth": 12.0,
+ "editorHide": true,
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false
+ }
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "markdown",
+ "editOnDblClick": true
+ },
+ "editorMode": "ace/mode/markdown",
+ "tableHide": false
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "HTML",
+ "data": "\u003cdiv class\u003d\"markdown-body\"\u003e\n\u003ch2\u003eCongratulations, it\u0026rsquo;s done.\u003c/h2\u003e\n\u003ch5\u003eYou can create your own notebook in \u0026lsquo;Notebook\u0026rsquo; menu. Good luck!\u003c/h5\u003e\n\u003c/div\u003e"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423836268492_216498320",
+ "id": "20150213-230428_1231780373",
+ "dateCreated": "Feb 13, 2015 11:04:28 PM",
+ "dateStarted": "Dec 17, 2016 3:30:24 PM",
+ "dateFinished": "Dec 17, 2016 3:30:29 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%md\n\nAbout bank data\n\n```\nCitation Request:\n This dataset is public available for research. The details are described in [Moro et al., 2011]. \n Please include this citation if you plan to use this database:\n\n [Moro et al., 2011] S. Moro, R. Laureano and P. Cortez. Using Data Mining for Bank Direct Marketing: An Application of the CRISP-DM Methodology. \n In P. Novais et al. (Eds.), Proceedings of the European Simulation and Modelling Conference - ESM\u00272011, pp. 117-121, Guimarães, Portugal, October, 2011. EUROSIS.\n\n Available at: [pdf] http://hdl.handle.net/1822/14838\n [bib] http://www3.dsi.uminho.pt/pcortez/bib/2011-esm-1.txt\n```",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:30:34 PM",
+ "config": {
+ "colWidth": 12.0,
+ "editorHide": true,
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false
+ }
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "markdown",
+ "editOnDblClick": true
+ },
+ "editorMode": "ace/mode/markdown",
+ "tableHide": false
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "HTML",
+ "data": "\u003cdiv class\u003d\"markdown-body\"\u003e\n\u003cp\u003eAbout bank data\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eCitation Request:\n This dataset is public available for research. The details are described in [Moro et al., 2011]. \n Please include this citation if you plan to use this database:\n\n [Moro et al., 2011] S. Moro, R. Laureano and P. Cortez. Using Data Mining for Bank Direct Marketing: An Application of the CRISP-DM Methodology. \n In P. Novais et al. (Eds.), Proceedings of the European Simulation and Modelling Conference - ESM\u0026#39;2011, pp. 117-121, Guimarães, Portugal, October, 2011. EUROSIS.\n\n Available at: [pdf] http://hdl.handle.net/1822/14838\n [bib] http://www3.dsi.uminho.pt/pcortez/bib/2011-esm-1.txt\n\u003c/code\u003e\u003c/pre\u003e\n\u003c/div\u003e"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1427420818407_872443482",
+ "id": "20150326-214658_12335843",
+ "dateCreated": "Mar 26, 2015 9:46:58 PM",
+ "dateStarted": "Dec 17, 2016 3:30:34 PM",
+ "dateFinished": "Dec 17, 2016 3:30:34 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "config": {},
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "apps": [],
+ "jobName": "paragraph_1435955447812_-158639899",
+ "id": "20150703-133047_853701097",
+ "dateCreated": "Jul 3, 2015 1:30:47 PM",
+ "status": "READY",
+ "progressUpdateIntervalMs": 500
+ }
+ ],
+ "name": "Zeppelin Tutorial/Basic Features (Spark)",
+ "id": "2A94M5J1Z",
+ "angularObjects": {
+ "2C73DY9P9:shared_process": []
+ },
+ "config": {
+ "looknfeel": "default"
+ },
+ "info": {}
+}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-server/src/test/resources/2A94M5J2Z/note.json
----------------------------------------------------------------------
diff --git a/zeppelin-server/src/test/resources/2A94M5J2Z/note.json b/zeppelin-server/src/test/resources/2A94M5J2Z/note.json
new file mode 100644
index 0000000..dd9a74d
--- /dev/null
+++ b/zeppelin-server/src/test/resources/2A94M5J2Z/note.json
@@ -0,0 +1,376 @@
+{
+ "paragraphs": [
+ {
+ "text": "%md\n## Welcome to Zeppelin.\n##### This is a live tutorial, you can run the code yourself. (Shift-Enter to Run)",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:32:15 PM",
+ "config": {
+ "colWidth": 12.0,
+ "editorHide": true,
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false,
+ "keys": [],
+ "values": [],
+ "groups": [],
+ "scatter": {}
+ }
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "markdown",
+ "editOnDblClick": true
+ },
+ "editorMode": "ace/mode/markdown",
+ "tableHide": false
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "HTML",
+ "data": "\u003cdiv class\u003d\"markdown-body\"\u003e\n\u003ch2\u003eWelcome to Zeppelin.\u003c/h2\u003e\n\u003ch5\u003eThis is a live tutorial, you can run the code yourself. (Shift-Enter to Run)\u003c/h5\u003e\n\u003c/div\u003e"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423836981412_-1007008116",
+ "id": "20150213-231621_168813393",
+ "dateCreated": "Feb 13, 2015 11:16:21 PM",
+ "dateStarted": "Dec 17, 2016 3:32:15 PM",
+ "dateFinished": "Dec 17, 2016 3:32:18 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "title": "Load data into table",
+ "text": "import org.apache.commons.io.IOUtils\nimport java.net.URL\nimport java.nio.charset.Charset\n\n// Zeppelin creates and injects sc (SparkContext) and sqlContext (HiveContext or SqlContext)\n// So you don\u0027t need create them manually\n\n// load bank data\nval bankText \u003d sc.parallelize(\n IOUtils.toString(\n new URL(\"https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv\"),\n Charset.forName(\"utf8\")).split(\"\\n\"))\n\ncase class Bank(age: Integer, job: String, marital: String, education: String, balance: Integer)\n\nval bank \u003d bankText.map(s \u003d\u003e s.split(\";\")).filter(s \u003d\u003e s(0) !\u003d \"\\\"age\\\"\").map(\n s \u003d\u003e Bank(s(0).toInt, \n s(1).replaceAll(\"\\\"\", \"\"),\n s(2).replaceAll(\"\\\"\", \"\"),\n s(3).replaceAll(\"\\\"\", \"\"),\n s(5).replaceAll(\"\\\"\", \"\").toInt\n )\n).toDF()\nbank.registerTempTable(\"bank\")",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:30:09 PM",
+ "config": {
+ "colWidth": 12.0,
+ "title": true,
+ "enabled": true,
+ "editorMode": "ace/mode/scala",
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false
+ }
+ }
+ ],
+ "editorSetting": {
+ "language": "scala",
+ "editOnDblClick": false
+ }
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TEXT",
+ "data": "import org.apache.commons.io.IOUtils\nimport java.net.URL\nimport java.nio.charset.Charset\nbankText: org.apache.spark.rdd.RDD[String] \u003d ParallelCollectionRDD[36] at parallelize at \u003cconsole\u003e:43\ndefined class Bank\nbank: org.apache.spark.sql.DataFrame \u003d [age: int, job: string ... 3 more fields]\nwarning: there were 1 deprecation warning(s); re-run with -deprecation for details\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423500779206_-1502780787",
+ "id": "20150210-015259_1403135953",
+ "dateCreated": "Feb 10, 2015 1:52:59 AM",
+ "dateStarted": "Dec 17, 2016 3:30:09 PM",
+ "dateFinished": "Dec 17, 2016 3:30:58 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%sql \nselect age, count(1) value\nfrom bank \nwhere age \u003c 30 \ngroup by age \norder by age",
+ "user": "anonymous",
+ "dateUpdated": "Mar 17, 2017 12:18:02 PM",
+ "config": {
+ "colWidth": 4.0,
+ "results": [
+ {
+ "graph": {
+ "mode": "multiBarChart",
+ "height": 366.0,
+ "optionOpen": false
+ },
+ "helium": {}
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "sql",
+ "editOnDblClick": false
+ },
+ "editorMode": "ace/mode/sql"
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TABLE",
+ "data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t20\n24\t24\n25\t44\n26\t77\n27\t94\n28\t103\n29\t97\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423500782552_-1439281894",
+ "id": "20150210-015302_1492795503",
+ "dateCreated": "Feb 10, 2015 1:53:02 AM",
+ "dateStarted": "Dec 17, 2016 3:30:13 PM",
+ "dateFinished": "Dec 17, 2016 3:31:04 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%sql \nselect age, count(1) value \nfrom bank \nwhere age \u003c ${maxAge\u003d30} \ngroup by age \norder by age",
+ "user": "anonymous",
+ "dateUpdated": "Mar 17, 2017 12:17:39 PM",
+ "config": {
+ "colWidth": 4.0,
+ "results": [
+ {
+ "graph": {
+ "mode": "multiBarChart",
+ "height": 294.0,
+ "optionOpen": false
+ },
+ "helium": {}
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "sql",
+ "editOnDblClick": false
+ },
+ "editorMode": "ace/mode/sql"
+ },
+ "settings": {
+ "params": {
+ "maxAge": "35"
+ },
+ "forms": {
+ "maxAge": {
+ "name": "maxAge",
+ "defaultValue": "30",
+ "hidden": false
+ }
+ }
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TABLE",
+ "data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t20\n24\t24\n25\t44\n26\t77\n27\t94\n28\t103\n29\t97\n30\t150\n31\t199\n32\t224\n33\t186\n34\t231\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423720444030_-1424110477",
+ "id": "20150212-145404_867439529",
+ "dateCreated": "Feb 12, 2015 2:54:04 PM",
+ "dateStarted": "Dec 17, 2016 3:30:58 PM",
+ "dateFinished": "Dec 17, 2016 3:31:07 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%sql \nselect age, count(1) value \nfrom bank \nwhere marital\u003d\"${marital\u003dsingle,single|divorced|married}\" \ngroup by age \norder by age",
+ "user": "anonymous",
+ "dateUpdated": "Mar 17, 2017 12:18:18 PM",
+ "config": {
+ "colWidth": 4.0,
+ "results": [
+ {
+ "graph": {
+ "mode": "stackedAreaChart",
+ "height": 280.0,
+ "optionOpen": false
+ },
+ "helium": {}
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "sql",
+ "editOnDblClick": false
+ },
+ "editorMode": "ace/mode/sql"
+ },
+ "settings": {
+ "params": {
+ "marital": "single"
+ },
+ "forms": {
+ "marital": {
+ "name": "marital",
+ "defaultValue": "single",
+ "options": [
+ {
+ "value": "single"
+ },
+ {
+ "value": "divorced"
+ },
+ {
+ "value": "married"
+ }
+ ],
+ "hidden": false
+ }
+ }
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "TABLE",
+ "data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t17\n24\t13\n25\t33\n26\t56\n27\t64\n28\t78\n29\t56\n30\t92\n31\t86\n32\t105\n33\t61\n34\t75\n35\t46\n36\t50\n37\t43\n38\t44\n39\t30\n40\t25\n41\t19\n42\t23\n43\t21\n44\t20\n45\t15\n46\t14\n47\t12\n48\t12\n49\t11\n50\t8\n51\t6\n52\t9\n53\t4\n55\t3\n56\t3\n57\t2\n58\t7\n59\t2\n60\t5\n66\t2\n69\t1\n"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423836262027_-210588283",
+ "id": "20150213-230422_1600658137",
+ "dateCreated": "Feb 13, 2015 11:04:22 PM",
+ "dateStarted": "Dec 17, 2016 3:31:05 PM",
+ "dateFinished": "Dec 17, 2016 3:31:09 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%md\n## Congratulations, it\u0027s done.\n##### You can create your own notebook in \u0027Notebook\u0027 menu. Good luck!",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:30:24 PM",
+ "config": {
+ "colWidth": 12.0,
+ "editorHide": true,
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false
+ }
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "markdown",
+ "editOnDblClick": true
+ },
+ "editorMode": "ace/mode/markdown",
+ "tableHide": false
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "HTML",
+ "data": "\u003cdiv class\u003d\"markdown-body\"\u003e\n\u003ch2\u003eCongratulations, it\u0026rsquo;s done.\u003c/h2\u003e\n\u003ch5\u003eYou can create your own notebook in \u0026lsquo;Notebook\u0026rsquo; menu. Good luck!\u003c/h5\u003e\n\u003c/div\u003e"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1423836268492_216498320",
+ "id": "20150213-230428_1231780373",
+ "dateCreated": "Feb 13, 2015 11:04:28 PM",
+ "dateStarted": "Dec 17, 2016 3:30:24 PM",
+ "dateFinished": "Dec 17, 2016 3:30:29 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "text": "%md\n\nAbout bank data\n\n```\nCitation Request:\n This dataset is public available for research. The details are described in [Moro et al., 2011]. \n Please include this citation if you plan to use this database:\n\n [Moro et al., 2011] S. Moro, R. Laureano and P. Cortez. Using Data Mining for Bank Direct Marketing: An Application of the CRISP-DM Methodology. \n In P. Novais et al. (Eds.), Proceedings of the European Simulation and Modelling Conference - ESM\u00272011, pp. 117-121, Guimarães, Portugal, October, 2011. EUROSIS.\n\n Available at: [pdf] http://hdl.handle.net/1822/14838\n [bib] http://www3.dsi.uminho.pt/pcortez/bib/2011-esm-1.txt\n```",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:30:34 PM",
+ "config": {
+ "colWidth": 12.0,
+ "editorHide": true,
+ "results": [
+ {
+ "graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false
+ }
+ }
+ ],
+ "enabled": true,
+ "editorSetting": {
+ "language": "markdown",
+ "editOnDblClick": true
+ },
+ "editorMode": "ace/mode/markdown",
+ "tableHide": false
+ },
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "results": {
+ "code": "SUCCESS",
+ "msg": [
+ {
+ "type": "HTML",
+ "data": "\u003cdiv class\u003d\"markdown-body\"\u003e\n\u003cp\u003eAbout bank data\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eCitation Request:\n This dataset is public available for research. The details are described in [Moro et al., 2011]. \n Please include this citation if you plan to use this database:\n\n [Moro et al., 2011] S. Moro, R. Laureano and P. Cortez. Using Data Mining for Bank Direct Marketing: An Application of the CRISP-DM Methodology. \n In P. Novais et al. (Eds.), Proceedings of the European Simulation and Modelling Conference - ESM\u0026#39;2011, pp. 117-121, Guimarães, Portugal, October, 2011. EUROSIS.\n\n Available at: [pdf] http://hdl.handle.net/1822/14838\n [bib] http://www3.dsi.uminho.pt/pcortez/bib/2011-esm-1.txt\n\u003c/code\u003e\u003c/pre\u003e\n\u003c/div\u003e"
+ }
+ ]
+ },
+ "apps": [],
+ "jobName": "paragraph_1427420818407_872443482",
+ "id": "20150326-214658_12335843",
+ "dateCreated": "Mar 26, 2015 9:46:58 PM",
+ "dateStarted": "Dec 17, 2016 3:30:34 PM",
+ "dateFinished": "Dec 17, 2016 3:30:34 PM",
+ "status": "FINISHED",
+ "progressUpdateIntervalMs": 500
+ },
+ {
+ "config": {},
+ "settings": {
+ "params": {},
+ "forms": {}
+ },
+ "apps": [],
+ "jobName": "paragraph_1435955447812_-158639899",
+ "id": "20150703-133047_853701097",
+ "dateCreated": "Jul 3, 2015 1:30:47 PM",
+ "status": "READY",
+ "progressUpdateIntervalMs": 500
+ }
+ ],
+ "name": "Zeppelin Tutorial/Basic Features (Spark)",
+ "id": "2A94M5J2Z",
+ "angularObjects": {
+ "2C73DY9P9:shared_process": []
+ },
+ "config": {
+ "looknfeel": "default"
+ },
+ "info": {}
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
index ff0ac62..72ea2ac 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
@@ -17,42 +17,16 @@
package org.apache.zeppelin.notebook;
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.Comparator;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.LinkedHashMap;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-
import com.google.common.base.Preconditions;
import com.google.common.base.Predicate;
import com.google.common.collect.FluentIterable;
import com.google.common.collect.Sets;
-import org.apache.zeppelin.interpreter.*;
-import org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry;
-import org.quartz.CronScheduleBuilder;
-import org.quartz.CronTrigger;
-import org.quartz.JobBuilder;
-import org.quartz.JobDetail;
-import org.quartz.JobExecutionContext;
-import org.quartz.JobExecutionException;
-import org.quartz.JobKey;
-import org.quartz.SchedulerException;
-import org.quartz.TriggerBuilder;
-import org.quartz.impl.StdSchedulerFactory;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.conf.ZeppelinConfiguration.ConfVars;
import org.apache.zeppelin.display.AngularObject;
import org.apache.zeppelin.display.AngularObjectRegistry;
+import org.apache.zeppelin.interpreter.*;
+import org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry;
import org.apache.zeppelin.notebook.repo.NotebookRepo;
import org.apache.zeppelin.notebook.repo.NotebookRepo.Revision;
import org.apache.zeppelin.notebook.repo.NotebookRepoSync;
@@ -61,6 +35,14 @@ import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.apache.zeppelin.search.SearchService;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.apache.zeppelin.user.Credentials;
+import org.quartz.*;
+import org.quartz.impl.StdSchedulerFactory;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.*;
+import java.util.concurrent.TimeUnit;
/**
* Collection of Notes.
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepo.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepo.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepo.java
new file mode 100644
index 0000000..6052e5f
--- /dev/null
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepo.java
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.notebook.repo;
+
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
+import org.apache.zeppelin.user.AuthenticationInfo;
+import org.eclipse.jgit.api.Git;
+import org.eclipse.jgit.api.PullCommand;
+import org.eclipse.jgit.api.PushCommand;
+import org.eclipse.jgit.api.RemoteAddCommand;
+import org.eclipse.jgit.api.errors.GitAPIException;
+import org.eclipse.jgit.transport.URIish;
+import org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+
+/**
+ * GitHub integration to store notebooks in a GitHub repository.
+ * It uses the same simple logic implemented in @see
+ * {@link org.apache.zeppelin.notebook.repo.GitNotebookRepo}
+ *
+ * The logic for updating the local repository from the remote repository is the following:
+ * - When the <code>GitHubNotebookRepo</code> is initialized
+ * - When pushing the changes to the remote repository
+ *
+ * The logic for updating the remote repository on GitHub from local repository is the following:
+ * - When commit the changes (saving the notebook)
+ */
+public class GitHubNotebookRepo extends GitNotebookRepo {
+ private static final Logger LOG = LoggerFactory.getLogger(GitNotebookRepo.class);
+ private ZeppelinConfiguration zeppelinConfiguration;
+ private Git git;
+
+ public GitHubNotebookRepo(ZeppelinConfiguration conf) throws IOException {
+ super(conf);
+
+ this.git = super.getGit();
+ this.zeppelinConfiguration = conf;
+
+ configureRemoteStream();
+ pullFromRemoteStream();
+ }
+
+ @Override
+ public Revision checkpoint(String pattern, String commitMessage, AuthenticationInfo subject) {
+ Revision revision = super.checkpoint(pattern, commitMessage, subject);
+
+ updateRemoteStream();
+
+ return revision;
+ }
+
+ private void configureRemoteStream() {
+ try {
+ LOG.debug("Setting up remote stream");
+ RemoteAddCommand remoteAddCommand = git.remoteAdd();
+ remoteAddCommand.setName(zeppelinConfiguration.getZeppelinNotebookGitRemoteOrigin());
+ remoteAddCommand.setUri(new URIish(zeppelinConfiguration.getZeppelinNotebookGitURL()));
+ remoteAddCommand.call();
+ } catch (GitAPIException e) {
+ LOG.error("Error configuring GitHub", e);
+ } catch (URISyntaxException e) {
+ LOG.error("Error in GitHub URL provided", e);
+ }
+ }
+
+ private void updateRemoteStream() {
+ LOG.debug("Updating remote stream");
+
+ pullFromRemoteStream();
+ pushToRemoteSteam();
+ }
+
+ private void pullFromRemoteStream() {
+ try {
+ LOG.debug("Pull latest changed from remote stream");
+ PullCommand pullCommand = git.pull();
+ pullCommand.setCredentialsProvider(
+ new UsernamePasswordCredentialsProvider(
+ zeppelinConfiguration.getZeppelinNotebookGitUsername(),
+ zeppelinConfiguration.getZeppelinNotebookGitAccessToken()
+ )
+ );
+
+ pullCommand.call();
+
+ } catch (GitAPIException e) {
+ LOG.error("Error when pulling latest changes from remote repository", e);
+ }
+ }
+
+ private void pushToRemoteSteam() {
+ try {
+ LOG.debug("Push latest changed from remote stream");
+ PushCommand pushCommand = git.push();
+ pushCommand.setCredentialsProvider(
+ new UsernamePasswordCredentialsProvider(
+ zeppelinConfiguration.getZeppelinNotebookGitUsername(),
+ zeppelinConfiguration.getZeppelinNotebookGitAccessToken()
+ )
+ );
+
+ pushCommand.call();
+ } catch (GitAPIException e) {
+ LOG.error("Error when pushing latest changes from remote repository", e);
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitNotebookRepo.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitNotebookRepo.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitNotebookRepo.java
index 21183da..2ac4c72 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitNotebookRepo.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/GitNotebookRepo.java
@@ -47,7 +47,8 @@ import com.google.common.collect.Lists;
*
* This impl intended to be simple and straightforward:
* - does not handle branches
- * - only basic local git file repo, no remote Github push\pull yet
+ * - only basic local git file repo, no remote Github push\pull. GitHub integration is
+ * implemented in @see {@link org.apache.zeppelin.notebook.repo.GitHubNotebookRepo}
*
* TODO(bzz): add default .gitignore
*/
@@ -177,7 +178,7 @@ public class GitNotebookRepo extends VFSNotebookRepo {
}
//DI replacements for Tests
- Git getGit() {
+ protected Git getGit() {
return git;
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepoTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepoTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepoTest.java
new file mode 100644
index 0000000..49a5cbd
--- /dev/null
+++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitHubNotebookRepoTest.java
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.notebook.repo;
+
+
+import com.google.common.base.Joiner;
+import org.apache.commons.io.FileUtils;
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
+import org.apache.zeppelin.interpreter.InterpreterFactory;
+import org.apache.zeppelin.notebook.Note;
+import org.apache.zeppelin.notebook.Paragraph;
+import org.apache.zeppelin.user.AuthenticationInfo;
+import org.eclipse.jgit.api.Git;
+import org.eclipse.jgit.api.errors.GitAPIException;
+import org.eclipse.jgit.internal.storage.file.FileRepository;
+import org.eclipse.jgit.lib.Repository;
+import org.eclipse.jgit.revwalk.RevCommit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.Iterator;
+
+import static org.mockito.Mockito.mock;
+
+/**
+ * This tests the remote Git tracking for notebooks. The tests uses two local Git repositories created locally
+ * to handle the tracking of Git actions (pushes and pulls). The repositories are:
+ * 1. The first repository is considered as a remote that mimics a remote GitHub directory
+ * 2. The second repository is considered as the local notebook repository
+ */
+public class GitHubNotebookRepoTest {
+ private static final Logger LOG = LoggerFactory.getLogger(GitNotebookRepoTest.class);
+
+ private static final String TEST_NOTE_ID = "2A94M5J1Z";
+
+ private File remoteZeppelinDir;
+ private File localZeppelinDir;
+ private String localNotebooksDir;
+ private String remoteNotebooksDir;
+ private ZeppelinConfiguration conf;
+ private GitHubNotebookRepo gitHubNotebookRepo;
+ private RevCommit firstCommitRevision;
+ private Git remoteGit;
+
+ @Before
+ public void setUp() throws Exception {
+ conf = ZeppelinConfiguration.create();
+
+ String remoteRepositoryPath = System.getProperty("java.io.tmpdir") + "/ZeppelinTestRemote_" +
+ System.currentTimeMillis();
+ String localRepositoryPath = System.getProperty("java.io.tmpdir") + "/ZeppelinTest_" +
+ System.currentTimeMillis();
+
+ // Create a fake remote notebook Git repository locally in another directory
+ remoteZeppelinDir = new File(remoteRepositoryPath);
+ remoteZeppelinDir.mkdirs();
+
+ // Create a local repository for notebooks
+ localZeppelinDir = new File(localRepositoryPath);
+ localZeppelinDir.mkdirs();
+
+ // Notebooks directory (for both the remote and local directories)
+ localNotebooksDir = Joiner.on(File.separator).join(localRepositoryPath, "notebook");
+ remoteNotebooksDir = Joiner.on(File.separator).join(remoteRepositoryPath, "notebook");
+
+ File notebookDir = new File(localNotebooksDir);
+ notebookDir.mkdirs();
+
+ // Copy the test notebook directory from the test/resources/2A94M5J1Z folder to the fake remote Git directory
+ String remoteTestNoteDir = Joiner.on(File.separator).join(remoteNotebooksDir, TEST_NOTE_ID);
+ FileUtils.copyDirectory(
+ new File(
+ GitHubNotebookRepoTest.class.getResource(
+ Joiner.on(File.separator).join("", TEST_NOTE_ID)
+ ).getFile()
+ ), new File(remoteTestNoteDir)
+ );
+
+ // Create the fake remote Git repository
+ Repository remoteRepository = new FileRepository(Joiner.on(File.separator).join(remoteNotebooksDir, ".git"));
+ remoteRepository.create();
+
+ remoteGit = new Git(remoteRepository);
+ remoteGit.add().addFilepattern(".").call();
+ firstCommitRevision = remoteGit.commit().setMessage("First commit from remote repository").call();
+
+ // Set the Git and Git configurations
+ System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_HOME.getVarName(), remoteZeppelinDir.getAbsolutePath());
+ System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_NOTEBOOK_DIR.getVarName(), notebookDir.getAbsolutePath());
+
+ // Set the GitHub configurations
+ System.setProperty(
+ ZeppelinConfiguration.ConfVars.ZEPPELIN_NOTEBOOK_STORAGE.getVarName(),
+ "org.apache.zeppelin.notebook.repo.GitHubNotebookRepo");
+ System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL.getVarName(),
+ remoteNotebooksDir + File.separator + ".git");
+ System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME.getVarName(), "token");
+ System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN.getVarName(),
+ "access-token");
+
+ // Create the Notebook repository (configured for the local repository)
+ gitHubNotebookRepo = new GitHubNotebookRepo(conf);
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ // Cleanup the temporary folders uses as Git repositories
+ File[] temporaryFolders = { remoteZeppelinDir, localZeppelinDir };
+
+ for(File temporaryFolder : temporaryFolders) {
+ if (!FileUtils.deleteQuietly(temporaryFolder))
+ LOG.error("Failed to delete {} ", temporaryFolder.getName());
+ }
+ }
+
+ @Test
+ /**
+ * Test the case when the Notebook repository is created, it pulls the latest changes from the remote repository
+ */
+ public void pullChangesFromRemoteRepositoryOnLoadingNotebook() throws IOException, GitAPIException {
+ NotebookRepo.Revision firstHistoryRevision = gitHubNotebookRepo.revisionHistory(TEST_NOTE_ID, null).get(0);
+
+ assert(this.firstCommitRevision.getName().equals(firstHistoryRevision.id));
+ }
+
+ @Test
+ /**
+ * Test the case when the check-pointing (add new files and commit) it also pulls the latest changes from the
+ * remote repository
+ */
+ public void pullChangesFromRemoteRepositoryOnCheckpointing() throws GitAPIException, IOException {
+ // Create a new commit in the remote repository
+ RevCommit secondCommitRevision = remoteGit.commit().setMessage("Second commit from remote repository").call();
+
+ // Add a new paragraph to the local repository
+ addParagraphToNotebook(TEST_NOTE_ID);
+
+ // Commit and push the changes to remote repository
+ NotebookRepo.Revision thirdCommitRevision = gitHubNotebookRepo.checkpoint(
+ TEST_NOTE_ID, "Third commit from local repository", null);
+
+ // Check all the commits as seen from the local repository. The commits are ordered chronologically. The last
+ // commit is the first in the commit logs.
+ Iterator<RevCommit> revisions = gitHubNotebookRepo.getGit().log().all().call().iterator();
+
+ revisions.next(); // The Merge `master` commit after pushing to the remote repository
+
+ assert(thirdCommitRevision.id.equals(revisions.next().getName())); // The local commit after adding the paragraph
+
+ // The second commit done on the remote repository
+ assert(secondCommitRevision.getName().equals(revisions.next().getName()));
+
+ // The first commit done on the remote repository
+ assert(firstCommitRevision.getName().equals(revisions.next().getName()));
+ }
+
+ @Test
+ /**
+ * Test the case when the check-pointing (add new files and commit) it pushes the local commits to the remote
+ * repository
+ */
+ public void pushLocalChangesToRemoteRepositoryOnCheckpointing() throws IOException, GitAPIException {
+ // Add a new paragraph to the local repository
+ addParagraphToNotebook(TEST_NOTE_ID);
+
+ // Commit and push the changes to remote repository
+ NotebookRepo.Revision secondCommitRevision = gitHubNotebookRepo.checkpoint(
+ TEST_NOTE_ID, "Second commit from local repository", null);
+
+ // Check all the commits as seen from the remote repository. The commits are ordered chronologically. The last
+ // commit is the first in the commit logs.
+ Iterator<RevCommit> revisions = remoteGit.log().all().call().iterator();
+
+ assert(secondCommitRevision.id.equals(revisions.next().getName())); // The local commit after adding the paragraph
+
+ // The first commit done on the remote repository
+ assert(firstCommitRevision.getName().equals(revisions.next().getName()));
+ }
+
+ private void addParagraphToNotebook(String noteId) throws IOException {
+ Note note = gitHubNotebookRepo.get(TEST_NOTE_ID, null);
+ note.setInterpreterFactory(mock(InterpreterFactory.class));
+ Paragraph paragraph = note.addNewParagraph(AuthenticationInfo.ANONYMOUS);
+ paragraph.setText("%md text");
+ gitHubNotebookRepo.save(note, null);
+ }
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/2be8f350/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitNotebookRepoTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitNotebookRepoTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitNotebookRepoTest.java
index 2276c25..72ea439 100644
--- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitNotebookRepoTest.java
+++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/GitNotebookRepoTest.java
@@ -70,9 +70,19 @@ public class GitNotebookRepoTest {
String testNoteDir = Joiner.on(File.separator).join(notebooksDir, TEST_NOTE_ID);
String testNoteDir2 = Joiner.on(File.separator).join(notebooksDir, TEST_NOTE_ID2);
- FileUtils.copyDirectory(new File(Joiner.on(File.separator).join("src", "test", "resources", TEST_NOTE_ID)),
- new File(testNoteDir));
- FileUtils.copyDirectory(new File(Joiner.on(File.separator).join("src", "test", "resources", TEST_NOTE_ID2)),
+ FileUtils.copyDirectory(
+ new File(
+ GitHubNotebookRepoTest.class.getResource(
+ Joiner.on(File.separator).join("", TEST_NOTE_ID)
+ ).getFile()
+ ),
+ new File(testNoteDir));
+ FileUtils.copyDirectory(
+ new File(
+ GitHubNotebookRepoTest.class.getResource(
+ Joiner.on(File.separator).join("", TEST_NOTE_ID2)
+ ).getFile()
+ ),
new File(testNoteDir2)
);