You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by zj...@apache.org on 2021/02/08 11:54:37 UTC
[zeppelin] branch master updated: [ZEPPELIN-5039]. Add document for
zeppelin sdk
This is an automated email from the ASF dual-hosted git repository.
zjffdu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 2f04fa8 [ZEPPELIN-5039]. Add document for zeppelin sdk
2f04fa8 is described below
commit 2f04fa891116fac77044bae9d62519728c2cc8b5
Author: Jeff Zhang <zj...@apache.org>
AuthorDate: Mon Sep 14 11:11:05 2020 +0800
[ZEPPELIN-5039]. Add document for zeppelin sdk
### What is this PR for?
Add 2 pages for zeppelin sdk document. One for client api, another for session api.
### What type of PR is it?
[Documentation ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-5039
### How should this be tested?
* No tests
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? NO
Author: Jeff Zhang <zj...@apache.org>
Closes #4027 from zjffdu/ZEPPELIN-5039 and squashes the following commits:
6bd563c31 [Jeff Zhang] [ZEPPELIN-5039]. Add document for zeppelin sdk
---
docs/_includes/themes/zeppelin/_navigation.html | 3 +
docs/index.md | 5 +-
docs/usage/zeppelin_sdk/client_api.md | 122 ++++++++++++++++++
docs/usage/zeppelin_sdk/session_api.md | 162 ++++++++++++++++++++++++
4 files changed, 291 insertions(+), 1 deletion(-)
diff --git a/docs/_includes/themes/zeppelin/_navigation.html b/docs/_includes/themes/zeppelin/_navigation.html
index 5f0eac4..e92a9c7 100644
--- a/docs/_includes/themes/zeppelin/_navigation.html
+++ b/docs/_includes/themes/zeppelin/_navigation.html
@@ -78,6 +78,9 @@
<li><a href="{{BASE_PATH}}/usage/rest_api/configuration.html">Configuration API</a></li>
<li><a href="{{BASE_PATH}}/usage/rest_api/credential.html">Credential API</a></li>
<li><a href="{{BASE_PATH}}/usage/rest_api/helium.html">Helium API</a></li>
+ <li class="title"><span>Zeppelin SDK</span></li>
+ <li><a href="{{BASE_PATH}}/usage/zeppelin_sdk/client_api.html">Client API</a></li>
+ <li><a href="{{BASE_PATH}}/usage/zeppelin_sdk/session_api.html">Session API</a></li>
</ul>
</li>
diff --git a/docs/index.md b/docs/index.md
index 36301d0..cb43433 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -84,7 +84,10 @@ limitations under the License.
* [Configuration API](./usage/rest_api/configuration.html)
* [Credential API](./usage/rest_api/credential.html)
* [Helium API](./usage/rest_api/helium.html)
-
+* Zeppelin SDK: Java API of Zeppelin Client
+ * [Client API](./usage/zeppelin_sdk/client_api.html)
+ * [Session API](./usage/zeppelin_sdk/session_api.html)
+
#### Setup
* Basics
* [How to Build Zeppelin](./setup/basics/how_to_build.html)
diff --git a/docs/usage/zeppelin_sdk/client_api.md b/docs/usage/zeppelin_sdk/client_api.md
new file mode 100644
index 0000000..d136bb8
--- /dev/null
+++ b/docs/usage/zeppelin_sdk/client_api.md
@@ -0,0 +1,122 @@
+---
+layout: page
+title: "Apache Zeppelin SDK - ZeppelinClient API"
+description: "This page contains Apache Zeppelin Client API."
+group: usage/zeppelin_sdk
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+{% include JB/setup %}
+
+# Apache Zeppelin SDK - Zeppelin Client API
+
+<div id="toc"></div>
+
+## Overview
+
+Zeppelin client api is a lower level java api which encapsulates Zeppelin's rest api so that you can easily integrate Zeppelin
+with your system. You can use zeppelin client api to do most of the things in notebook ui in a programmatic way, such as create/delete note/paragraph,
+run note/paragraph and etc.
+
+
+## How to use Zeppelin Client API
+
+The entry point of zeppelin client api is class `ZeppelinClient`. All the operations is via this class, e.g. in the following example, we use `ZeppelinClient` to run the spark tutorial note programmatically.
+
+{% highlight java %}
+ClientConfig clientConfig = new ClientConfig("http://localhost:8080");
+ZeppelinClient zClient = new ZeppelinClient(clientConfig);
+
+String zeppelinVersion = zClient.getVersion();
+System.out.println("Zeppelin version: " + zeppelinVersion);
+
+// execute note 2A94M5J1Z paragraph by paragraph
+try {
+ ParagraphResult paragraphResult = zClient.executeParagraph("2A94M5J1Z", "20150210-015259_1403135953");
+ System.out.println("Execute the 1st spark tutorial paragraph, paragraph result: " + paragraphResult);
+
+ paragraphResult = zClient.executeParagraph("2A94M5J1Z", "20150210-015302_1492795503");
+ System.out.println("Execute the 2nd spark tutorial paragraph, paragraph result: " + paragraphResult);
+
+ Map<String, String> parameters = new HashMap<>();
+ parameters.put("maxAge", "40");
+ paragraphResult = zClient.executeParagraph("2A94M5J1Z", "20150212-145404_867439529", parameters);
+ System.out.println("Execute the 3rd spark tutorial paragraph, paragraph result: " + paragraphResult);
+
+ parameters = new HashMap<>();
+ parameters.put("marital", "married");
+ paragraphResult = zClient.executeParagraph("2A94M5J1Z", "20150213-230422_1600658137", parameters);
+ System.out.println("Execute the 4th spark tutorial paragraph, paragraph result: " + paragraphResult);
+} finally {
+ // you need to stop interpreter explicitly if you are running paragraph separately.
+ zClient.stopInterpreter("2A94M5J1Z", "spark");
+}
+{% endhighlight %}
+
+Here we list some importance apis of ZeppelinClient, for the completed api, please refer its javadoc.
+
+{% highlight java %}
+
+public String createNote(String notePath) throws Exception
+
+public void deleteNote(String noteId) throws Exception
+
+public NoteResult executeNote(String noteId) throws Exception
+
+public NoteResult executeNote(String noteId,
+ Map<String, String> parameters) throws Exception
+
+public NoteResult queryNoteResult(String noteId) throws Exception
+
+public NoteResult submitNote(String noteId) throws Exception
+
+public NoteResult submitNote(String noteId,
+ Map<String, String> parameters) throws Exception
+
+public NoteResult waitUntilNoteFinished(String noteId) throws Exception
+
+public String addParagraph(String noteId,
+ String title,
+ String text) throws Exception
+
+public void updateParagraph(String noteId,
+ String paragraphId,
+ String title,
+ String text) throws Exception
+
+public ParagraphResult executeParagraph(String noteId,
+ String paragraphId,
+ String sessionId,
+ Map<String, String> parameters) throws Exception
+
+public ParagraphResult submitParagraph(String noteId,
+ String paragraphId,
+ String sessionId,
+ Map<String, String> parameters) throws Exception
+
+public void cancelParagraph(String noteId, String paragraphId)
+
+public ParagraphResult queryParagraphResult(String noteId, String paragraphId)
+
+public ParagraphResult waitUtilParagraphFinish(String noteId, String paragraphId)
+
+{% endhighlight %}
+
+
+## Examples
+
+For more detailed usage of zeppelin client api, you can check the examples in module `zeppelin-client-examples`
+
+* [ZeppelinClientExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/ZeppelinClientExample.java)]
+* [ZeppelinClientExample2](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/ZeppelinClientExample2.java)]
diff --git a/docs/usage/zeppelin_sdk/session_api.md b/docs/usage/zeppelin_sdk/session_api.md
new file mode 100644
index 0000000..6597c1d
--- /dev/null
+++ b/docs/usage/zeppelin_sdk/session_api.md
@@ -0,0 +1,162 @@
+---
+layout: page
+title: "Apache Zeppelin SDK - Session API"
+description: "This page contains Apache Zeppelin SDK - Session API."
+group: usage/zeppelin_sdk
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+{% include JB/setup %}
+
+# Apache Zeppelin SDK - Session API
+
+<div id="toc"></div>
+
+## Overview
+
+Session api is a high level api for zeppelin. There's no zeppelin concept (note, paragraph) in this api. The most important thing is a ZSession which represent a running interpreter process.
+It is pretty to create a ZSession and its api is very straightforward, we can see a concret examples below.
+
+
+## How to use ZSession
+
+It is very easy to create a ZSession, you need to provide ClientConfig, interpreter and also you can customize your ZSession by specificy its interpreter properties.
+
+After you can create ZSession, you need to start it before running any code.
+ZSession's lifecycle is under your control, you need to call stop method expclitly, otherwise the interpreter processs will keep running.
+
+{% highlight java %}
+ZSession session = null;
+try {
+ ClientConfig clientConfig = new ClientConfig("http://localhost:8080");
+ Map<String, String> intpProperties = new HashMap<>();
+ intpProperties.put("spark.master", "local[*]");
+
+ session = ZSession.builder()
+ .setClientConfig(clientConfig)
+ .setInterpreter("spark")
+ .setIntpProperties(intpProperties)
+ .build();
+
+ session.start();
+ System.out.println("Spark Web UI: " + session.getWeburl());
+
+ // scala (single result)
+ ExecuteResult result = session.execute("println(sc.version)");
+ System.out.println("Spark Version: " + result.getResults().get(0).getData());
+
+ // scala (multiple result)
+ result = session.execute("println(sc.version)\n" +
+ "val df = spark.createDataFrame(Seq((1,\"a\"), (2,\"b\")))\n" +
+ "z.show(df)");
+
+ // The first result is text output
+ System.out.println("Result 1: type: " + result.getResults().get(0).getType() +
+ ", data: " + result.getResults().get(0).getData() );
+ // The second result is table output
+ System.out.println("Result 2: type: " + result.getResults().get(1).getType() +
+ ", data: " + result.getResults().get(1).getData() );
+ System.out.println("Spark Job Urls:\n" + StringUtils.join(result.getJobUrls(), "\n"));
+
+ // error output
+ result = session.execute("1/0");
+ System.out.println("Result status: " + result.getStatus() +
+ ", data: " + result.getResults().get(0).getData());
+
+ // pyspark
+ result = session.execute("pyspark", "df = spark.createDataFrame([(1,'a'),(2,'b')])\n" +
+ "df.registerTempTable('df')\n" +
+ "df.show()");
+ System.out.println("PySpark dataframe: " + result.getResults().get(0).getData());
+
+ // matplotlib
+ result = session.execute("ipyspark", "%matplotlib inline\n" +
+ "import matplotlib.pyplot as plt\n" +
+ "plt.plot([1,2,3,4])\n" +
+ "plt.ylabel('some numbers')\n" +
+ "plt.show()");
+ System.out.println("Matplotlib result, type: " + result.getResults().get(0).getType() +
+ ", data: " + result.getResults().get(0).getData());
+
+ // sparkr
+ result = session.execute("r", "df <- as.DataFrame(faithful)\nhead(df)");
+ System.out.println("Sparkr dataframe: " + result.getResults().get(0).getData());
+
+ // spark sql
+ result = session.execute("sql", "select * from df");
+ System.out.println("Spark Sql dataframe: " + result.getResults().get(0).getData());
+
+ // spark invalid sql
+ result = session.execute("sql", "select * from unknown_table");
+ System.out.println("Result status: " + result.getStatus() +
+ ", data: " + result.getResults().get(0).getData());
+} catch (Exception e) {
+ e.printStackTrace();
+} finally {
+ if (session != null) {
+ try {
+ session.stop();
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+ }
+}
+{% endhighlight %}
+
+Here's a list of apis of `ZSession`.
+
+{% highlight java %}
+public void start() throws Exception
+
+public void start(MessageHandler messageHandler) throws Exception
+
+public void stop() throws Exception
+
+public ExecuteResult execute(String code) throws Exception
+
+public ExecuteResult execute(String subInterpreter,
+ Map<String, String> localProperties,
+ String code,
+ StatementMessageHandler messageHandler) throws Exception
+
+public ExecuteResult submit(String code) throws Exception
+
+public ExecuteResult submit(String subInterpreter,
+ Map<String, String> localProperties,
+ String code,
+ StatementMessageHandler messageHandler) throws Exception
+
+public void cancel(String statementId) throws Exception
+
+public ExecuteResult queryStatement(String statementId) throws Exception
+
+public ExecuteResult waitUntilFinished(String statementId) throws Exception
+
+{% endhighlight %}
+
+
+
+## Examples
+
+For more detailed usage of session api, you can check the examples in module `zeppelin-client-examples`
+
+* [SparkExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/SparkExample.java)
+* [SparkAdvancedExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/SparkAdvancedExample.java)
+* [FlinkExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/FlinkExample.java)
+* [FlinkAdvancedExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/FlinkAdvancedExample.java)
+* [FlinkAdvancedExample2](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/FlinkAdvancedExample2.java)
+* [HiveExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/Hive.java)
+* [PythonExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/PythonExample.java)
+* [RExample](https://github.com/apache/zeppelin/blob/master/zeppelin-client-examples/src/main/java/org/apache/zeppelin/client/examples/RExample.java)
+