You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mo...@apache.org on 2016/11/24 18:26:46 UTC
svn commit: r1771200 [3/3] - in /zeppelin/site/docs/0.7.0-SNAPSHOT: ./
assets/themes/zeppelin/img/docs-img/
assets/themes/zeppelin/img/screenshots/ development/ displaysystem/
install/ interpreter/ manual/ quickstart/ rest-api/ security/ storage/
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json Thu Nov 24 18:26:45 2016
@@ -105,7 +105,7 @@
"/install/install.html": {
"title": "Quick Start",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Quick StartWelcome to Apache Zeppelin! On this page are instructions to help you get started.InstallationApache Zeppelin officially supports and is tested on the following environments: Name Value Oracle JDK 1.7 (set JAVA_HOME) OS Mac OSX Ubuntu 14.X CentOS 6.X Windows 7 Pro SP1 Downloading Binary PackageTwo binary packages are available on the Apache Zeppelin Download Page. Only difference between
these two binaries is interpreters are included in the package file.Package with all interpreters.Just unpack it in a directory of your choice and you&#39;re ready to go.Package with net-install interpreters.Unpack and follow install additional interpreters to install interpreters. If you&#39;re unsure, just run ./bin/install-interpreter.sh --all and install all interpreters.Starting Apache Zeppelin from the Command LineStarting Apache ZeppelinOn all unix like platforms:bin/zeppelin-daemon.sh startIf you are on Windows:binzeppelin.cmdAfter Zeppelin has started successfully, go to http://localhost:8080 with your web browser.Stopping Zeppelinbin/zeppelin-daemon.sh stopNext StepsCongratulations, you have successfully installed Apache Zeppelin! Here are few steps you might find useful:New to Apache Zeppelin...For an in-depth overview, head to Explore Apache Zeppelin UI.And then, try run tutorial notebook in your Zeppelin.And see how to change configurations like port number, etc
.Zeppelin with Apache Spark ...To know more about deep integration with Apache Spark, check Spark Interpreter.Zeppelin with JDBC data sources ...Check JDBC Interpreter to know more about configure and uses multiple JDBC data sources.Zeppelin with Python ...Check Python interpreter to know more about Matplotlib, Pandas integration.Multi-user environment ...Turn on authentication.Manage your notebook permission.For more informations, go to More -&gt; Security section.Other useful informations ...Learn how Display System works.Use Service Manager to start Zeppelin.If you&#39;re using previous version please see Upgrade Zeppelin version.Apache Zeppelin ConfigurationYou can configure Apache Zeppelin with either environment variables in conf/zeppelin-env.sh (confzeppelin-env.cmd for Windows) or Java properties in conf/zeppelin-site.xml. If both are defined, then the environment variables will take priority. zeppelin-env.sh zeppelin-site.xml Default value Description
ZEPPELIN_PORT zeppelin.server.port 8080 Zeppelin server port ZEPPELIN_SSL_PORT zeppelin.server.ssl.port 8443 Zeppelin Server ssl port (used when ssl environment/property is set to true) ZEPPELIN_MEM N/A -Xmx1024m -XX:MaxPermSize=512m JVM mem options ZEPPELIN_INTP_MEM N/A ZEPPELIN_MEM JVM mem options for interpreter process ZEPPELIN_JAVA_OPTS N/A JVM options ZEPPELIN_ALLOWED_ORIGINS zeppelin.server.allowed.origins * Enables a way to specify a ',' separated list of allowed origins for REST and websockets. i.e. http://localhost:8080 N/A zeppelin.anonymous.allowed true The anonymous user is allowed by default. ZEPPELIN_SERVER_CONTEXT_PATH zeppelin.server.context.path / Context path of the web application ZEPPELIN_SSL zeppelin.ssl false ZEPPELIN_SSL_CLIENT_AUTH zeppelin.ssl.client.auth false ZEPPELIN_SSL_KEYS
TORE_PATH zeppelin.ssl.keystore.path keystore ZEPPELIN_SSL_KEYSTORE_TYPE zeppelin.ssl.keystore.type JKS ZEPPELIN_SSL_KEYSTORE_PASSWORD zeppelin.ssl.keystore.password ZEPPELIN_SSL_KEY_MANAGER_PASSWORD zeppelin.ssl.key.manager.password ZEPPELIN_SSL_TRUSTSTORE_PATH zeppelin.ssl.truststore.path ZEPPELIN_SSL_TRUSTSTORE_TYPE zeppelin.ssl.truststore.type ZEPPELIN_SSL_TRUSTSTORE_PASSWORD zeppelin.ssl.truststore.password ZEPPELIN_NOTEBOOK_HOMESCREEN zeppelin.notebook.homescreen Display note IDs on the Apache Zeppelin homescreen i.e. 2A94M5J1Z ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE zeppelin.notebook.homescreen.hide false Hide the note ID set by ZEPPELIN_NOTEBOOK_HOMESCREEN on the Apache Zeppelin homescreen. For the further information, please read Customize your Zeppelin homepage. ZEPPELIN_WAR_TEMPDIR zeppelin.war.tempdir webapps Loca
tion of the jetty temporary directory ZEPPELIN_NOTEBOOK_DIR zeppelin.notebook.dir notebook The root directory where notebook directories are saved ZEPPELIN_NOTEBOOK_S3_BUCKET zeppelin.notebook.s3.bucket zeppelin S3 Bucket where notebook files will be saved ZEPPELIN_NOTEBOOK_S3_USER zeppelin.notebook.s3.user user User name of an S3 bucketi.e. bucket/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_S3_ENDPOINT zeppelin.notebook.s3.endpoint s3.amazonaws.com Endpoint for the bucket ZEPPELIN_NOTEBOOK_S3_KMS_KEY_ID zeppelin.notebook.s3.kmsKeyID AWS KMS Key ID to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_S3_EMP zeppelin.notebook.s3.encryptionMaterialsProvider Class name of a custom S3 encryption materials provider implementation to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_AZURE_CONNECTION_STRING zeppelin.notebook.azure.connectionString The
Azure storage account connection stringi.e. DefaultEndpointsProtocol=https;AccountName=&lt;accountName&gt;;AccountKey=&lt;accountKey&gt; ZEPPELIN_NOTEBOOK_AZURE_SHARE zeppelin.notebook.azure.share zeppelin Azure Share where the notebook files will be saved ZEPPELIN_NOTEBOOK_AZURE_USER zeppelin.notebook.azure.user user Optional user name of an Azure file sharei.e. share/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_STORAGE zeppelin.notebook.storage org.apache.zeppelin.notebook.repo.VFSNotebookRepo Comma separated list of notebook storage locations ZEPPELIN_NOTEBOOK_ONE_WAY_SYNC zeppelin.notebook.one.way.sync false If there are multiple notebook storage locations, should we treat the first one as the only source of truth? ZEPPELIN_INTERPRETERS zeppelin.interpreters org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkSq
lInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.shell.ShellInterpreter, ... Comma separated interpreter configurations [Class] NOTE: This property is deprecated since Zeppelin-0.6.0 and will not be supported from Zeppelin-0.7.0 on. ZEPPELIN_INTERPRETER_DIR zeppelin.interpreter.dir interpreter Interpreter directory ZEPPELIN_WEBSOCKET_MAX_TEXT_MESSAGE_SIZE zeppelin.websocket.max.text.message.size 1024000 Size (in characters) of the maximum text message that can be received by websocket. Start Apache Zeppelin with a service managerNote : The below description was written based on Ubuntu Linux.Apache Zeppelin can be auto-started as a service with an init script, using a service manager like upstart.This is an example upstart script saved as /etc/init/zeppelin.confThis allows the service to be managed with commands such assudo service zeppelin start sudo service zep
pelin stop sudo service zeppelin restartOther service managers could use a similar approach with the upstart argument passed to the zeppelin-daemon.sh script.bin/zeppelin-daemon.sh upstartzeppelin.confdescription &quot;zeppelin&quot;start on (local-filesystems and net-device-up IFACE!=lo)stop on shutdown# Respawn the process on unexpected terminationrespawn# respawn the job up to 7 times within a 5 second period.# If the job exceeds these values, it will be stopped and marked as failed.respawn limit 7 5# zeppelin was installed in /usr/share/zeppelin in this examplechdir /usr/share/zeppelinexec bin/zeppelin-daemon.sh upstartBuilding from SourceIf you want to build from source instead of using binary package, follow the instructions here.",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Quick StartWelcome to Apache Zeppelin! On this page are instructions to help you get started.InstallationApache Zeppelin officially supports and is tested on the following environments: Name Value Oracle JDK 1.7 (set JAVA_HOME) OS Mac OSX Ubuntu 14.X CentOS 6.X Windows 7 Pro SP1 Downloading Binary PackageTwo binary packages are available on the Apache Zeppelin Download Page. Only difference between
these two binaries is interpreters are included in the package file.Package with all interpreters.Just unpack it in a directory of your choice and you&#39;re ready to go.Package with net-install interpreters.Unpack and follow install additional interpreters to install interpreters. If you&#39;re unsure, just run ./bin/install-interpreter.sh --all and install all interpreters.Starting Apache Zeppelin from the Command LineStarting Apache ZeppelinOn all unix like platforms:bin/zeppelin-daemon.sh startIf you are on Windows:binzeppelin.cmdAfter Zeppelin has started successfully, go to http://localhost:8080 with your web browser.Stopping Zeppelinbin/zeppelin-daemon.sh stopNext StepsCongratulations, you have successfully installed Apache Zeppelin! Here are few steps you might find useful:New to Apache Zeppelin...For an in-depth overview, head to Explore Apache Zeppelin UI.And then, try run tutorial notebook in your Zeppelin.And see how to change configurations like port number, etc
.Zeppelin with Apache Spark ...To know more about deep integration with Apache Spark, check Spark Interpreter.Zeppelin with JDBC data sources ...Check JDBC Interpreter to know more about configure and uses multiple JDBC data sources.Zeppelin with Python ...Check Python interpreter to know more about Matplotlib, Pandas, Conda/Docker environment integration.Multi-user environment ...Turn on authentication.Manage your notebook permission.For more informations, go to More -&gt; Security section.Other useful informations ...Learn how Display System works.Use Service Manager to start Zeppelin.If you&#39;re using previous version please see Upgrade Zeppelin version.Apache Zeppelin ConfigurationYou can configure Apache Zeppelin with either environment variables in conf/zeppelin-env.sh (confzeppelin-env.cmd for Windows) or Java properties in conf/zeppelin-site.xml. If both are defined, then the environment variables will take priority. zeppelin-env.sh zeppelin-site.xml Def
ault value Description ZEPPELIN_PORT zeppelin.server.port 8080 Zeppelin server port ZEPPELIN_SSL_PORT zeppelin.server.ssl.port 8443 Zeppelin Server ssl port (used when ssl environment/property is set to true) ZEPPELIN_MEM N/A -Xmx1024m -XX:MaxPermSize=512m JVM mem options ZEPPELIN_INTP_MEM N/A ZEPPELIN_MEM JVM mem options for interpreter process ZEPPELIN_JAVA_OPTS N/A JVM options ZEPPELIN_ALLOWED_ORIGINS zeppelin.server.allowed.origins * Enables a way to specify a ',' separated list of allowed origins for REST and websockets. i.e. http://localhost:8080 N/A zeppelin.anonymous.allowed true The anonymous user is allowed by default. ZEPPELIN_SERVER_CONTEXT_PATH zeppelin.server.context.path / Context path of the web application ZEPPELIN_SSL zeppelin.ssl false ZEPPELIN_SSL_CLIENT_AUTH zeppelin.ssl.client.auth false
ZEPPELIN_SSL_KEYSTORE_PATH zeppelin.ssl.keystore.path keystore ZEPPELIN_SSL_KEYSTORE_TYPE zeppelin.ssl.keystore.type JKS ZEPPELIN_SSL_KEYSTORE_PASSWORD zeppelin.ssl.keystore.password ZEPPELIN_SSL_KEY_MANAGER_PASSWORD zeppelin.ssl.key.manager.password ZEPPELIN_SSL_TRUSTSTORE_PATH zeppelin.ssl.truststore.path ZEPPELIN_SSL_TRUSTSTORE_TYPE zeppelin.ssl.truststore.type ZEPPELIN_SSL_TRUSTSTORE_PASSWORD zeppelin.ssl.truststore.password ZEPPELIN_NOTEBOOK_HOMESCREEN zeppelin.notebook.homescreen Display note IDs on the Apache Zeppelin homescreen i.e. 2A94M5J1Z ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE zeppelin.notebook.homescreen.hide false Hide the note ID set by ZEPPELIN_NOTEBOOK_HOMESCREEN on the Apache Zeppelin homescreen. For the further information, please read Customize your Zeppelin homepage. ZEPPELIN_WAR_TEMPDIR zeppelin.war.
tempdir webapps Location of the jetty temporary directory ZEPPELIN_NOTEBOOK_DIR zeppelin.notebook.dir notebook The root directory where notebook directories are saved ZEPPELIN_NOTEBOOK_S3_BUCKET zeppelin.notebook.s3.bucket zeppelin S3 Bucket where notebook files will be saved ZEPPELIN_NOTEBOOK_S3_USER zeppelin.notebook.s3.user user User name of an S3 bucketi.e. bucket/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_S3_ENDPOINT zeppelin.notebook.s3.endpoint s3.amazonaws.com Endpoint for the bucket ZEPPELIN_NOTEBOOK_S3_KMS_KEY_ID zeppelin.notebook.s3.kmsKeyID AWS KMS Key ID to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_S3_EMP zeppelin.notebook.s3.encryptionMaterialsProvider Class name of a custom S3 encryption materials provider implementation to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_AZURE_CONNECTION_STRING zeppelin.notebook.azure.c
onnectionString The Azure storage account connection stringi.e. DefaultEndpointsProtocol=https;AccountName=&lt;accountName&gt;;AccountKey=&lt;accountKey&gt; ZEPPELIN_NOTEBOOK_AZURE_SHARE zeppelin.notebook.azure.share zeppelin Azure Share where the notebook files will be saved ZEPPELIN_NOTEBOOK_AZURE_USER zeppelin.notebook.azure.user user Optional user name of an Azure file sharei.e. share/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_STORAGE zeppelin.notebook.storage org.apache.zeppelin.notebook.repo.VFSNotebookRepo Comma separated list of notebook storage locations ZEPPELIN_NOTEBOOK_ONE_WAY_SYNC zeppelin.notebook.one.way.sync false If there are multiple notebook storage locations, should we treat the first one as the only source of truth? ZEPPELIN_NOTEBOOK_PUBLIC zeppelin.notebook.public true Make notebook public (set only `owners`) by default when created/imported. If set t
o `false` will add `user` to `readers` and `writers` as well, making it private and invisible to other users unless permissions are granted. ZEPPELIN_INTERPRETERS zeppelin.interpreters org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.shell.ShellInterpreter, ... Comma separated interpreter configurations [Class] NOTE: This property is deprecated since Zeppelin-0.6.0 and will not be supported from Zeppelin-0.7.0 on. ZEPPELIN_INTERPRETER_DIR zeppelin.interpreter.dir interpreter Interpreter directory ZEPPELIN_WEBSOCKET_MAX_TEXT_MESSAGE_SIZE zeppelin.websocket.max.text.message.size 1024000 Size (in characters) of the maximum text message that can be received by websocket. Start Apache Zeppelin with a service managerNote : The below de
scription was written based on Ubuntu Linux.Apache Zeppelin can be auto-started as a service with an init script, using a service manager like upstart.This is an example upstart script saved as /etc/init/zeppelin.confThis allows the service to be managed with commands such assudo service zeppelin start sudo service zeppelin stop sudo service zeppelin restartOther service managers could use a similar approach with the upstart argument passed to the zeppelin-daemon.sh script.bin/zeppelin-daemon.sh upstartzeppelin.confdescription &quot;zeppelin&quot;start on (local-filesystems and net-device-up IFACE!=lo)stop on shutdown# Respawn the process on unexpected terminationrespawn# respawn the job up to 7 times within a 5 second period.# If the job exceeds these values, it will be stopped and marked as failed.respawn limit 7 5# zeppelin was installed in /usr/share/zeppelin in this examplechdir /usr/share/zeppelinexec bin/zeppelin-daemon.sh upstartBuilding from SourceIf you want to b
uild from source instead of using binary package, follow the instructions here.",
"url": " /install/install.html",
"group": "install",
"excerpt": "This page will help you get started and will guide you through installing Apache Zeppelin, running it in the command line and configuring options."
@@ -116,7 +116,7 @@
"/install/spark_cluster_mode.html": {
"title": "Apache Zeppelin on Spark cluster mode",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Apache Zeppelin on Spark Cluster ModeOverviewApache Spark has supported three cluster manager types(Standalone, Apache Mesos and Hadoop YARN) so far.This document will guide you how you can build and configure the environment on 3 types of Spark cluster manager with Apache Zeppelin using Docker scripts.So install docker on the machine first.Spark standalone modeSpark standalone is a simple cluster manager included with Spark that m
akes it easy to set up a cluster.You can simply set up Spark standalone environment with below steps. Note : Since Apache Zeppelin and Spark use same 8080 port for their web UI, you might need to change zeppelin.server.port in conf/zeppelin-site.xml.1. Build Docker fileYou can find docker script files under scripts/docker/spark-cluster-managers.cd $ZEPPELIN_HOME/scripts/docker/spark-cluster-managers/spark_standalonedocker build -t &quot;spark_standalone&quot; .2. Run dockerdocker run -it -p 8080:8080 -p 7077:7077 -p 8888:8888 -p 8081:8081 -h sparkmaster --name spark_standalone spark_standalone bash; 3. Configure Spark interpreter in ZeppelinSet Spark master as spark://&lt;hostname&gt;:7077 in Zeppelin Interpreters setting page.4. Run Zeppelin with Spark interpreterAfter running single paragraph with Spark interpreter in Zeppelin, browse https://&lt;hostname&gt;:8080 and check whether Spark cluster is running well or not.You can also simply verify that Spark i
s running well in Docker with below command.ps -ef | grep sparkSpark on YARN modeYou can simply set up Spark on YARN docker environment with below steps.Note : Since Apache Zeppelin and Spark use same 8080 port for their web UI, you might need to change zeppelin.server.port in conf/zeppelin-site.xml.1. Build Docker fileYou can find docker script files under scripts/docker/spark-cluster-managers.cd $ZEPPELIN_HOME/scripts/docker/spark-cluster-managers/spark_yarndocker build -t &quot;spark_yarn&quot; .2. Run dockerdocker run -it -p 5000:5000 -p 9000:9000 -p 9001:9001 -p 8088:8088 -p 8042:8042 -p 8030:8030 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8080:8080 -p 7077:7077 -p 8888:8888 -p 8081:8081 -p 50010:50010 -p 50075:50075 -p 50020:50020 -p 50070:50070 --name spark_yarn -h sparkmaster spark_yarn bash;3. Verify running Spark on YARN.You can simply verify the processes of Spark and YARN are running well in Docker with below command.ps -efYou can also check ea
ch application web UI for HDFS on http://&lt;hostname&gt;:50070/, YARN on http://&lt;hostname&gt;:8088/cluster and Spark on http://&lt;hostname&gt;:8080/.4. Configure Spark interpreter in ZeppelinSet following configurations to conf/zeppelin-env.sh.export MASTER=yarn-clientexport HADOOP_CONF_DIR=[your_hadoop_conf_path]export SPARK_HOME=[your_spark_home_path]HADOOP_CONF_DIR(Hadoop configuration path) is defined in /scripts/docker/spark-cluster-managers/spark_yarn_cluster/hdfs_conf.Don&#39;t forget to set Spark master as yarn-client in Zeppelin Interpreters setting page like below.5. Run Zeppelin with Spark interpreterAfter running a single paragraph with Spark interpreter in Zeppelin, browse http://&lt;hostname&gt;:8088/cluster/apps and check Zeppelin application is running well or not.Spark on Mesos modeYou can simply set up Spark on Mesos docker environment with below steps.1. Build Docker filecd $ZEPPELIN_HOME/scripts/docker/spark-cluster-manage
rs/spark_mesosdocker build -t &quot;spark_mesos&quot; .2. Run dockerdocker run --net=host -it -p 8080:8080 -p 7077:7077 -p 8888:8888 -p 8081:8081 -p 8082:8082 -p 5050:5050 -p 5051:5051 -p 4040:4040 -h sparkmaster --name spark_mesos spark_mesos bash;3. Verify running Spark on Mesos.You can simply verify the processes of Spark and Mesos are running well in Docker with below command.ps -efYou can also check each application web UI for Mesos on http://&lt;hostname&gt;:5050/cluster and Spark on http://&lt;hostname&gt;:8080/.4. Configure Spark interpreter in Zeppelinexport MASTER=mesos://127.0.1.1:5050export MESOS_NATIVE_JAVA_LIBRARY=[PATH OF libmesos.so]export SPARK_HOME=[PATH OF SPARK HOME]Don&#39;t forget to set Spark master as mesos://127.0.1.1:5050 in Zeppelin Interpreters setting page like below.5. Run Zeppelin with Spark interpreterAfter running a single paragraph with Spark interpreter in Zeppelin, browse http://&lt;hostname&gt;:5050/#/framework
s and check Zeppelin application is running well or not.",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Apache Zeppelin on Spark Cluster ModeOverviewApache Spark has supported three cluster manager types(Standalone, Apache Mesos and Hadoop YARN) so far.This document will guide you how you can build and configure the environment on 3 types of Spark cluster manager with Apache Zeppelin using Docker scripts.So install docker on the machine first.Spark standalone modeSpark standalone is a simple cluster manager included with Spark that m
akes it easy to set up a cluster.You can simply set up Spark standalone environment with below steps.Note : Since Apache Zeppelin and Spark use same 8080 port for their web UI, you might need to change zeppelin.server.port in conf/zeppelin-site.xml.1. Build Docker fileYou can find docker script files under scripts/docker/spark-cluster-managers.cd $ZEPPELIN_HOME/scripts/docker/spark-cluster-managers/spark_standalonedocker build -t &quot;spark_standalone&quot; .2. Run dockerdocker run -it -p 8080:8080 -p 7077:7077 -p 8888:8888 -p 8081:8081 -h sparkmaster --name spark_standalone spark_standalone bash;Note that sparkmaster hostname used here to run docker container should be defined in your /etc/hosts.3. Configure Spark interpreter in ZeppelinSet Spark master as spark://&lt;hostname&gt;:7077 in Zeppelin Interpreters setting page.4. Run Zeppelin with Spark interpreterAfter running single paragraph with Spark interpreter in Zeppelin, browse https://&lt;hostname&gt;
:8080 and check whether Spark cluster is running well or not.You can also simply verify that Spark is running well in Docker with below command.ps -ef | grep sparkSpark on YARN modeYou can simply set up Spark on YARN docker environment with below steps.Note : Since Apache Zeppelin and Spark use same 8080 port for their web UI, you might need to change zeppelin.server.port in conf/zeppelin-site.xml.1. Build Docker fileYou can find docker script files under scripts/docker/spark-cluster-managers.cd $ZEPPELIN_HOME/scripts/docker/spark-cluster-managers/spark_yarn_clusterdocker build -t &quot;spark_yarn&quot; .2. Run dockerdocker run -it -p 5000:5000 -p 9000:9000 -p 9001:9001 -p 8088:8088 -p 8042:8042 -p 8030:8030 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8080:8080 -p 7077:7077 -p 8888:8888 -p 8081:8081 -p 50010:50010 -p 50075:50075 -p 50020:50020 -p 50070:50070 --name spark_yarn -h sparkmaster spark_yarn bash;Note that sparkmaster hostname used here to run doc
ker container should be defined in your /etc/hosts.3. Verify running Spark on YARN.You can simply verify the processes of Spark and YARN are running well in Docker with below command.ps -efYou can also check each application web UI for HDFS on http://&lt;hostname&gt;:50070/, YARN on http://&lt;hostname&gt;:8088/cluster and Spark on http://&lt;hostname&gt;:8080/.4. Configure Spark interpreter in ZeppelinSet following configurations to conf/zeppelin-env.sh.export MASTER=yarn-clientexport HADOOP_CONF_DIR=[your_hadoop_conf_path]export SPARK_HOME=[your_spark_home_path]HADOOP_CONF_DIR(Hadoop configuration path) is defined in /scripts/docker/spark-cluster-managers/spark_yarn_cluster/hdfs_conf.Don&#39;t forget to set Spark master as yarn-client in Zeppelin Interpreters setting page like below.5. Run Zeppelin with Spark interpreterAfter running a single paragraph with Spark interpreter in Zeppelin, browse http://&lt;hostname&gt;:8088/cluster/apps and check
Zeppelin application is running well or not.Spark on Mesos modeYou can simply set up Spark on Mesos docker environment with below steps.1. Build Docker filecd $ZEPPELIN_HOME/scripts/docker/spark-cluster-managers/spark_mesosdocker build -t &quot;spark_mesos&quot; .2. Run dockerdocker run --net=host -it -p 8080:8080 -p 7077:7077 -p 8888:8888 -p 8081:8081 -p 8082:8082 -p 5050:5050 -p 5051:5051 -p 4040:4040 -h sparkmaster --name spark_mesos spark_mesos bash;Note that sparkmaster hostname used here to run docker container should be defined in your /etc/hosts.3. Verify running Spark on Mesos.You can simply verify the processes of Spark and Mesos are running well in Docker with below command.ps -efYou can also check each application web UI for Mesos on http://&lt;hostname&gt;:5050/cluster and Spark on http://&lt;hostname&gt;:8080/.4. Configure Spark interpreter in Zeppelinexport MASTER=mesos://127.0.1.1:5050export MESOS_NATIVE_JAVA_LIBRARY=[PATH OF libmesos.so]expo
rt SPARK_HOME=[PATH OF SPARK HOME]Don&#39;t forget to set Spark master as mesos://127.0.1.1:5050 in Zeppelin Interpreters setting page like below.5. Run Zeppelin with Spark interpreterAfter running a single paragraph with Spark interpreter in Zeppelin, browse http://&lt;hostname&gt;:5050/#/frameworks and check Zeppelin application is running well or not.",
"url": " /install/spark_cluster_mode.html",
"group": "install",
"excerpt": "This document will guide you how you can build and configure the environment on 3 types of Spark cluster manager(Standalone, Hadoop Yarn, Apache Mesos) with Apache Zeppelin using docker scripts."
@@ -127,7 +127,7 @@
"/install/upgrade.html": {
"title": "Manual Zeppelin version upgrade procedure",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Manual upgrade procedure for ZeppelinBasically, newer version of Zeppelin works with previous version notebook directory and configurations.So, copying notebook and conf directory should be enough.InstructionsStop Zeppelinbin/zeppelin-daemon.sh stopCopy your notebook and conf directory into a backup directoryDownload newer version of Zeppelin and Install. See Install page.Copy backup notebook and conf directory into newer version o
f Zeppelin notebook and conf directoryStart Zeppelinbin/zeppelin-daemon.sh startMigration GuideUpgrading from Zeppelin 0.6 to 0.7From 0.7, we don&#39;t use ZEPPELIN_JAVA_OPTS as default value of ZEPPELIN_INTP_JAVA_OPTS and also the same for ZEPPELIN_MEM/ZEPPELIN_INTP_MEM. If user want to configure the jvm opts of interpreter process, please set ZEPPELIN_INTP_JAVA_OPTS and ZEPPELIN_INTP_MEM explicitly. If you don&#39;t set ZEPPELIN_INTP_MEM, Zeppelin will set it to -Xms1024m -Xmx1024m -XX:MaxPermSize=512m by default.Mapping from %jdbc(prefix) to %prefix is no longer available. Instead, you can use %[interpreter alias] with multiple interpreter setttings on GUI.Usage of ZEPPELIN_PORT is not supported in ssl mode. Instead use ZEPPELIN_SSL_PORT to configure the ssl port. Value from ZEPPELIN_PORT is used only when ZEPPELIN_SSL is set to false.The support on Spark 1.1.x to 1.3.x is deprecated.",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Manual upgrade procedure for ZeppelinBasically, newer version of Zeppelin works with previous version notebook directory and configurations.So, copying notebook and conf directory should be enough.InstructionsStop Zeppelinbin/zeppelin-daemon.sh stopCopy your notebook and conf directory into a backup directoryDownload newer version of Zeppelin and Install. See Install page.Copy backup notebook and conf directory into newer version o
f Zeppelin notebook and conf directoryStart Zeppelinbin/zeppelin-daemon.sh startMigration GuideUpgrading from Zeppelin 0.6 to 0.7From 0.7, we don&#39;t use ZEPPELIN_JAVA_OPTS as default value of ZEPPELIN_INTP_JAVA_OPTS and also the same for ZEPPELIN_MEM/ZEPPELIN_INTP_MEM. If user want to configure the jvm opts of interpreter process, please set ZEPPELIN_INTP_JAVA_OPTS and ZEPPELIN_INTP_MEM explicitly. If you don&#39;t set ZEPPELIN_INTP_MEM, Zeppelin will set it to -Xms1024m -Xmx1024m -XX:MaxPermSize=512m by default.Mapping from %jdbc(prefix) to %prefix is no longer available. Instead, you can use %[interpreter alias] with multiple interpreter setttings on GUI.Usage of ZEPPELIN_PORT is not supported in ssl mode. Instead use ZEPPELIN_SSL_PORT to configure the ssl port. Value from ZEPPELIN_PORT is used only when ZEPPELIN_SSL is set to false.The support on Spark 1.1.x to 1.3.x is deprecated.From 0.7, we uses pegdown as the markdown.parser.type option for the %md interpreter. Ren
dered markdown might be different from what you expected",
"url": " /install/upgrade.html",
"group": "install",
"excerpt": "This document will guide you through a procedure of manual upgrade your Apache Zeppelin instance to a newer version. Apache Zeppelin keeps backward compatibility for the notebook file format."
@@ -281,7 +281,7 @@
"/interpreter/jdbc.html": {
"title": "Generic JDBC Interpreter for Apache Zeppelin",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Generic JDBC Interpreter for Apache ZeppelinOverviewJDBC interpreter lets you create a JDBC connection to any data sources seamlessly. By now, it has been tested with: Postgresql - JDBC Driver Mysql - JDBC Driver MariaDB - JDBC Driver Redshift - JDBC Driver Apache Hive - JDBC Driver Apache Phoenix itself is a JDBC driver
Apache Drill - JDBC Driver Apache Tajo - JDBC Driver If you are using other databases not in the above list, please feel free to share your use case. It would be helpful to improve the functionality of JDBC interpreter.Create a new JDBC InterpreterFirst, click + Create button at the top-right corner in the interpreter setting page.Fill Interpreter name field with whatever you want to use as the alias(e.g. mysql, mysql2, hive, redshift, and etc..). Please note that this alias will be used as %interpreter_name to call the interpreter in the paragraph. Then select jdbc as an Interpreter group. The default driver of JDBC interpreter is set as PostgreSQL. It means Zeppelin includes PostgreSQL driver jar in itself.So you don&#39;t need to add any dependencies(e.g. the artifact name or path for PostgreSQL driver jar) for PostgreSQL connection.The JDBC interpreter properties are defined by default like below. Name Default Value Descrip
tion common.max_count 1000 The maximun number of SQL result to display default.driver org.postgresql.Driver JDBC Driver Name default.password The JDBC user password default.url jdbc:postgresql://localhost:5432/ The URL for JDBC default.user gpadmin The JDBC user name If you want to connect other databases such as Mysql, Redshift and Hive, you need to edit the property values. The below example is for Mysql connection.The last step is Dependency Setting. Since Zeppelin only includes PostgreSQL driver jar by default, you need to add each driver&#39;s maven coordinates or JDBC driver&#39;s jar file path for the other databases.That&#39;s it. You can find more JDBC connection setting examples(Mysql, MariaDB, Redshift, Apache Hive, Apache Phoenix, and Apache Tajo) in this section.More propertiesThere are more JDBC interpreter properties you can specify like below. Property Name Description commo
n.max_result Max number of SQL result to display to prevent the browser overload. This is common properties for all connections zeppelin.jdbc.auth.type Types of authentications' methods supported are SIMPLE, and KERBEROS zeppelin.jdbc.principal The principal name to load from the keytab zeppelin.jdbc.keytab.location The path to the keytab file You can also add more properties by using this method.For example, if a connection needs a schema parameter, it would have to add the property as follows: name value default.schema schema_name Binding JDBC interpter to notebookTo bind the interpreters created in the interpreter setting page, click the gear icon at the top-right corner.Select(blue) or deselect(white) the interpreter buttons depending on your use cases. If you need to use more than one interpreter in the notebook, activate several buttons.Don&#39;t forget to click Save button, or you will face Interpreter *** is not f
ound error.How to useRun the paragraph with JDBC interpreterTo test whether your databases and Zeppelin are successfully connected or not, type %jdbc_interpreter_name(e.g. %mysql) at the top of the paragraph and run show databases.%jdbc_interpreter_nameshow databasesIf the paragraph is FINISHED without any errors, a new paragraph will be automatically added after the previous one with %jdbc_interpreter_name.So you don&#39;t need to type this prefix in every paragraphs&#39; header.Apply Zeppelin Dynamic FormsYou can leverage Zeppelin Dynamic Form inside your queries. You can use both the text input and select form parametrization features.%jdbc_interpreter_nameSELECT name, country, performerFROM demo.performersWHERE name=&#39;{{performer=Sheryl Crow|Doof|Fanfarlo|Los Paranoia}}&#39;ExamplesHere are some examples you can refer to. Including the below connectors, you can connect every databases as long as it can be configured with it&#39;s JDBC driver.PostgresProper
ties Name Value default.driver org.postgresql.Driver default.url jdbc:postgresql://localhost:5432/ default.user mysql_user default.password mysql_password Postgres JDBC Driver DocsDependencies Artifact Excludes org.postgresql:postgresql:9.4.1211 Maven Repository: org.postgresql:postgresqlMysqlProperties Name Value default.driver com.mysql.jdbc.Driver default.url jdbc:mysql://localhost:3306/ default.user mysql_user default.password mysql_password Mysql JDBC Driver DocsDependencies Artifact Excludes mysql:mysql-connector-java:5.1.38 Maven Repository: mysql:mysql-connector-javaMariaDBProperties Name Value default.driver org.mariadb.jdbc.Driver default.url jdbc:mariadb://localhost:3306 default.user mariadb_user default.password mariadb_password MariaDB JDBC Driver DocsDependencies Artifact Exclu
des org.mariadb.jdbc:mariadb-java-client:1.5.4 Maven Repository: org.mariadb.jdbc:mariadb-java-clientRedshiftProperties Name Value default.driver com.amazon.redshift.jdbc42.Driver default.url jdbc:redshift://your-redshift-instance-address.redshift.amazonaws.com:5439/your-database default.user redshift_user default.password redshift_password AWS Redshift JDBC Driver DocsDependencies Artifact Excludes com.amazonaws:aws-java-sdk-redshift:1.11.51 Maven Repository: com.amazonaws:aws-java-sdk-redshiftApache HiveProperties Name Value default.driver org.apache.hive.jdbc.HiveDriver default.url jdbc:hive2://localhost:10000 default.user hive_user default.password hive_password Apache Hive 1 JDBC Driver DocsApache Hive 2 JDBC Driver DocsDependencies Artifact Excludes org.apache.hive:hive-jdbc:0.14.0 org.apache.hadoop:hadoop-common:2.6.0
Maven Repository : org.apache.hive:hive-jdbcApache PhoenixPhoenix supports thick and thin connection types:Thick client is faster, but must connect directly to ZooKeeper and HBase RegionServers.Thin client has fewer dependencies and connects through a Phoenix Query Server instance.Use the appropriate default.driver, default.url, and the dependency artifact for your connection type.Thick client connectionProperties Name Value default.driver org.apache.phoenix.jdbc.PhoenixDriver default.url jdbc:phoenix:localhost:2181:/hbase-unsecure default.user phoenix_user default.password phoenix_password Dependencies Artifact Excludes org.apache.phoenix:phoenix-core:4.4.0-HBase-1.0 Maven Repository: org.apache.phoenix:phoenix-coreThin client connectionProperties Name Value default.driver org.apache.phoenix.queryserver.client.Driver default.url jdbc:phoenix:thin:url=http://localhost:8765;serialization
=PROTOBUF default.user phoenix_user default.password phoenix_password DependenciesBefore Adding one of the below dependencies, check the Phoenix version first. Artifact Excludes Description org.apache.phoenix:phoenix-server-client:4.7.0-HBase-1.1 For Phoenix 4.7 org.apache.phoenix:phoenix-queryserver-client:4.8.0-HBase-1.2 For Phoenix 4.8+ Maven Repository: org.apache.phoenix:phoenix-queryserver-clientApache TajoProperties Name Value default.driver org.apache.tajo.jdbc.TajoDriver default.url jdbc:tajo://localhost:26002/default Apache Tajo JDBC Driver DocsDependencies Artifact Excludes org.apache.tajo:tajo-jdbc:0.11.0 Maven Repository: org.apache.tajo:tajo-jdbcBug reportingIf you find a bug using JDBC interpreter, please create a JIRA ticket.",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Generic JDBC Interpreter for Apache ZeppelinOverviewJDBC interpreter lets you create a JDBC connection to any data sources seamlessly. By now, it has been tested with: Postgresql - JDBC Driver Mysql - JDBC Driver MariaDB - JDBC Driver Redshift - JDBC Driver Apache Hive - JDBC Driver Apache Phoenix itself is a JDBC driver
Apache Drill - JDBC Driver Apache Tajo - JDBC Driver If you are using other databases not in the above list, please feel free to share your use case. It would be helpful to improve the functionality of JDBC interpreter.Create a new JDBC InterpreterFirst, click + Create button at the top-right corner in the interpreter setting page.Fill Interpreter name field with whatever you want to use as the alias(e.g. mysql, mysql2, hive, redshift, and etc..). Please note that this alias will be used as %interpreter_name to call the interpreter in the paragraph. Then select jdbc as an Interpreter group. The default driver of JDBC interpreter is set as PostgreSQL. It means Zeppelin includes PostgreSQL driver jar in itself.So you don&#39;t need to add any dependencies(e.g. the artifact name or path for PostgreSQL driver jar) for PostgreSQL connection.The JDBC interpreter properties are defined by default like below. Name Default Value Descrip
tion common.max_count 1000 The maximun number of SQL result to display default.driver org.postgresql.Driver JDBC Driver Name default.password The JDBC user password default.url jdbc:postgresql://localhost:5432/ The URL for JDBC default.user gpadmin The JDBC user name If you want to connect other databases such as Mysql, Redshift and Hive, you need to edit the property values.You can also use Credential for JDBC authentication.If default.user and default.password properties are deleted(using X button) for database connection in the interpreter setting page,the JDBC interpreter will get the account information from Credential.The below example is for Mysql connection.The last step is Dependency Setting. Since Zeppelin only includes PostgreSQL driver jar by default, you need to add each driver&#39;s maven coordinates or JDBC driver&#39;s jar file path for the other databases.That&#39;s it. You can find more
JDBC connection setting examples(Mysql, MariaDB, Redshift, Apache Hive, Apache Phoenix, and Apache Tajo) in this section.More propertiesThere are more JDBC interpreter properties you can specify like below. Property Name Description common.max_result Max number of SQL result to display to prevent the browser overload. This is common properties for all connections zeppelin.jdbc.auth.type Types of authentications' methods supported are SIMPLE, and KERBEROS zeppelin.jdbc.principal The principal name to load from the keytab zeppelin.jdbc.keytab.location The path to the keytab file You can also add more properties by using this method.For example, if a connection needs a schema parameter, it would have to add the property as follows: name value default.schema schema_name Binding JDBC interpter to notebookTo bind the interpreters created in the interpreter setting page, click the gear icon at the top-right corner.Sel
ect(blue) or deselect(white) the interpreter buttons depending on your use cases. If you need to use more than one interpreter in the notebook, activate several buttons.Don&#39;t forget to click Save button, or you will face Interpreter *** is not found error.How to useRun the paragraph with JDBC interpreterTo test whether your databases and Zeppelin are successfully connected or not, type %jdbc_interpreter_name(e.g. %mysql) at the top of the paragraph and run show databases.%jdbc_interpreter_nameshow databasesIf the paragraph is FINISHED without any errors, a new paragraph will be automatically added after the previous one with %jdbc_interpreter_name.So you don&#39;t need to type this prefix in every paragraphs&#39; header.Apply Zeppelin Dynamic FormsYou can leverage Zeppelin Dynamic Form inside your queries. You can use both the text input and select form parametrization features.%jdbc_interpreter_nameSELECT name, country, performerFROM demo.performersWHERE name=&#
39;{{performer=Sheryl Crow|Doof|Fanfarlo|Los Paranoia}}&#39;ExamplesHere are some examples you can refer to. Including the below connectors, you can connect every databases as long as it can be configured with it&#39;s JDBC driver.PostgresProperties Name Value default.driver org.postgresql.Driver default.url jdbc:postgresql://localhost:5432/ default.user mysql_user default.password mysql_password Postgres JDBC Driver DocsDependencies Artifact Excludes org.postgresql:postgresql:9.4.1211 Maven Repository: org.postgresql:postgresqlMysqlProperties Name Value default.driver com.mysql.jdbc.Driver default.url jdbc:mysql://localhost:3306/ default.user mysql_user default.password mysql_password Mysql JDBC Driver DocsDependencies Artifact Excludes mysql:mysql-connector-java:5.1.38 Maven Repository: mysql:mysql-connector-javaMariaDBProperties Na
me Value default.driver org.mariadb.jdbc.Driver default.url jdbc:mariadb://localhost:3306 default.user mariadb_user default.password mariadb_password MariaDB JDBC Driver DocsDependencies Artifact Excludes org.mariadb.jdbc:mariadb-java-client:1.5.4 Maven Repository: org.mariadb.jdbc:mariadb-java-clientRedshiftProperties Name Value default.driver com.amazon.redshift.jdbc42.Driver default.url jdbc:redshift://your-redshift-instance-address.redshift.amazonaws.com:5439/your-database default.user redshift_user default.password redshift_password AWS Redshift JDBC Driver DocsDependencies Artifact Excludes com.amazonaws:aws-java-sdk-redshift:1.11.51 Maven Repository: com.amazonaws:aws-java-sdk-redshiftApache HiveProperties Name Value default.driver org.apache.hive.jdbc.HiveDriver default.url jdbc:hive2://localhost:10000 default
.user hive_user default.password hive_password Apache Hive 1 JDBC Driver DocsApache Hive 2 JDBC Driver DocsDependencies Artifact Excludes org.apache.hive:hive-jdbc:0.14.0 org.apache.hadoop:hadoop-common:2.6.0 Maven Repository : org.apache.hive:hive-jdbcApache PhoenixPhoenix supports thick and thin connection types:Thick client is faster, but must connect directly to ZooKeeper and HBase RegionServers.Thin client has fewer dependencies and connects through a Phoenix Query Server instance.Use the appropriate default.driver, default.url, and the dependency artifact for your connection type.Thick client connectionProperties Name Value default.driver org.apache.phoenix.jdbc.PhoenixDriver default.url jdbc:phoenix:localhost:2181:/hbase-unsecure default.user phoenix_user default.password phoenix_password Dependencies Artifact Excludes org.apache.phoenix:phoenix-core:4.4.0-HBase-1.0
Maven Repository: org.apache.phoenix:phoenix-coreThin client connectionProperties Name Value default.driver org.apache.phoenix.queryserver.client.Driver default.url jdbc:phoenix:thin:url=http://localhost:8765;serialization=PROTOBUF default.user phoenix_user default.password phoenix_password DependenciesBefore Adding one of the below dependencies, check the Phoenix version first. Artifact Excludes Description org.apache.phoenix:phoenix-server-client:4.7.0-HBase-1.1 For Phoenix 4.7 org.apache.phoenix:phoenix-queryserver-client:4.8.0-HBase-1.2 For Phoenix 4.8+ Maven Repository: org.apache.phoenix:phoenix-queryserver-clientApache TajoProperties Name Value default.driver org.apache.tajo.jdbc.TajoDriver default.url jdbc:tajo://localhost:26002/default Apache Tajo JDBC Driver DocsDependencies Artifact Excludes org.apache.tajo:tajo-jdbc:0.11.0 Maven Rep
ository: org.apache.tajo:tajo-jdbcBug reportingIf you find a bug using JDBC interpreter, please create a JIRA ticket.",
"url": " /interpreter/jdbc.html",
"group": "interpreter",
"excerpt": "Generic JDBC Interpreter lets you create a JDBC connection to any data source. You can use Postgres, MySql, MariaDB, Redshift, Apache Hive, Apache Phoenix, Apache Drill and Apache Tajo using JDBC interpreter."
@@ -336,7 +336,7 @@
"/interpreter/markdown.html": {
"title": "Markdown Interpreter for Apache Zeppelin",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Markdown Interpreter for Apache ZeppelinOverviewMarkdown is a plain text formatting syntax designed so that it can be converted to HTML.Apache Zeppelin uses markdown4j and pegdown as markdown parsers.In Zeppelin notebook, you can use %md in the beginning of a paragraph to invoke the Markdown interpreter and generate static html from Markdown plain text.In Zeppelin, Markdown interpreter is enabled by default and uses the markdown4j
parser.Configuration Name Default Value Description markdown.parser.type markdown4j Markdown Parser Type. Available values: markdown4j, pegdown. ExampleThe following example demonstrates the basic usage of Markdown in a Zeppelin notebook.Mathematical expressionMarkdown interpreter leverages %html display system internally. That means you can mix mathematical expressions with markdown syntax. For more information, please see Mathematical Expression section.Markdown4j Parsermarkdown4j parser provides YUML and Websequence extensions Pegdown Parserpegdown parser provides github flavored markdown.",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Markdown Interpreter for Apache ZeppelinOverviewMarkdown is a plain text formatting syntax designed so that it can be converted to HTML.Apache Zeppelin uses pegdown and markdown4j as markdown parsers.In Zeppelin notebook, you can use %md in the beginning of a paragraph to invoke the Markdown interpreter and generate static html from Markdown plain text.In Zeppelin, Markdown interpreter is enabled by default and uses the pegdown par
ser.ExampleThe following example demonstrates the basic usage of Markdown in a Zeppelin notebook.Mathematical expressionMarkdown interpreter leverages %html display system internally. That means you can mix mathematical expressions with markdown syntax. For more information, please see Mathematical Expression section.Configuration Name Default Value Description markdown.parser.type pegdown Markdown Parser Type. Available values: pegdown, markdown4j. Pegdown Parserpegdown parser provides github flavored markdown.pegdown parser provides YUML and Websequence plugins also. Markdown4j ParserSince pegdown parser is more accurate and provides much more markdown syntaxmarkdown4j option might be removed later. But keep this parser for the backward compatibility.",
"url": " /interpreter/markdown.html",
"group": "interpreter",
"excerpt": "Markdown is a plain text formatting syntax designed so that it can be converted to HTML. Apache Zeppelin uses markdown4j."
@@ -369,7 +369,7 @@
"/interpreter/python.html": {
"title": "Python 2 & 3 Interpreter for Apache Zeppelin",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Python 2 &amp; 3 Interpreter for Apache ZeppelinConfiguration Property Default Description zeppelin.python python Path of the already installed Python binary (could be python2 or python3). If python is not in your $PATH you can set the absolute directory (example : /usr/bin/python) zeppelin.python.maxResult 1000 Max number of dataframe rows to display. Enabling Python InterpreterIn a
notebook, to enable the Python interpreter, click on the Gear icon and select PythonUsing the Python InterpreterIn a paragraph, use %python to select the Python interpreter and then input all commands.The interpreter can only work if you already have python installed (the interpreter doesn&#39;t bring it own python binaries).To access the help, type help()Python modulesThe interpreter can use all modules already installed (with pip, easy_install...)Using Zeppelin Dynamic FormsYou can leverage Zeppelin Dynamic Form inside your Python code.Zeppelin Dynamic Form can only be used if py4j Python library is installed in your system. If not, you can install it with pip install py4j.Example : %python### Input formprint (z.input(&quot;f1&quot;,&quot;defaultValue&quot;))### Select formprint (z.select(&quot;f1&quot;,[(&quot;o1&quot;,&quot;1&quot;),(&quot;o2&quot;,&quot;2&quot;)],&quot;2&quot;))### Checkbox formprint(&quot
;&quot;.join(z.checkbox(&quot;f3&quot;, [(&quot;o1&quot;,&quot;1&quot;), (&quot;o2&quot;,&quot;2&quot;)],[&quot;1&quot;])))Matplotlib integrationThe python interpreter can display matplotlib figures inline automatically using the pyplot module:%pythonimport matplotlib.pyplot as pltplt.plot([1, 2, 3])This is the recommended method for using matplotlib from within a Zeppelin notebook. The output of this command will by default be converted to HTML by implicitly making use of the %html magic. Additional configuration can be achieved using the builtin z.configure_mpl() method. For example, z.configure_mpl(width=400, height=300, fmt=&#39;svg&#39;)plt.plot([1, 2, 3])Will produce a 400x300 image in SVG format, which by default are normally 600x400 and PNG respectively. In the future, another option called angular can be used to make it possible to update a plot produced from one paragraph directly from another (the output will be
%angular instead of %html). However, this feature is already available in the pyspark interpreter. More details can be found in the included &quot;Zeppelin Tutorial: Python - matplotlib basic&quot; tutorial notebook. If Zeppelin cannot find the matplotlib backend files (which should usually be found in $ZEPPELIN_HOME/interpreter/lib/python) in your PYTHONPATH, then the backend will automatically be set to agg, and the (otherwise deprecated) instructions below can be used for more limited inline plotting.If you are unable to load the inline backend, use z.show(plt): python%pythonimport matplotlib.pyplot as pltplt.figure()(.. ..)z.show(plt)plt.close()The z.show() function can take optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).%pythonz.show(plt, width=&#39;50px&#39;)z.show(plt, height=&#39;150px&#39;, fmt=&#39;svg&#39;)Pandas integrationApache Zeppelin Table Display System provides built
-in data visualization capabilities. Python interpreter leverages it to visualize Pandas DataFrames though similar z.show() API, same as with Matplotlib integration.Example:import pandas as pdrates = pd.read_csv(&quot;bank.csv&quot;, sep=&quot;;&quot;)z.show(rates)SQL over Pandas DataFramesThere is a convenience %python.sql interpreter that matches Apache Spark experience in Zeppelin and enables usage of SQL language to query Pandas DataFrames and visualization of results though built-in Table Display System.Pre-requestsPandas pip install pandasPandaSQL pip install -U pandasqlIn case default binded interpreter is Python (first in the interpreter list, under the Gear Icon), you can just use it as %sql i.efirst paragraphimport pandas as pdrates = pd.read_csv(&quot;bank.csv&quot;, sep=&quot;;&quot;)next paragraph%sqlSELECT * FROM rates WHERE age &lt; 40Otherwise it can be referred to as %python.sqlTechnical descriptionFor in-depth technical details o
n current implementation please refer to python/README.md.Some features not yet implemented in the Python InterpreterInterrupt a paragraph execution (cancel() method) is currently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter. A JIRA ticket (ZEPPELIN-893) is opened to implement this feature in a next release of the interpreter.Progression bar in webUI (getProgress() method) is currently not implemented.Code-completion is currently not implemented.",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Python 2 &amp; 3 Interpreter for Apache ZeppelinConfiguration Property Default Description zeppelin.python python Path of the already installed Python binary (could be python2 or python3). If python is not in your $PATH you can set the absolute directory (example : /usr/bin/python) zeppelin.python.maxResult 1000 Max number of dataframe rows to display. Enabling Python InterpreterIn a
notebook, to enable the Python interpreter, click on the Gear icon and select PythonUsing the Python InterpreterIn a paragraph, use %python to select the Python interpreter and then input all commands.The interpreter can only work if you already have python installed (the interpreter doesn&#39;t bring it own python binaries).To access the help, type help()Python environmentsDefaultBy default, PythonInterpreter will use python command defined in zeppelin.python property to run python process.The interpreter can use all modules already installed (with pip, easy_install...)CondaConda is an package management system and environment management system for python.%python.conda interpreter lets you change between environments.UsageList your environments%python.condaActivate an environment%python.conda activate [ENVIRONMENT_NAME]Deactivate%python.conda deactivateDocker%python.docker interpreter allows PythonInterpreter creates python process in a specified docker container.UsageActivate
an environment%python.docker activate [Repository]%python.docker activate [Repository:Tag]%python.docker activate [Image Id]Deactivate%python.docker deactivateExample# activate latest tensorflow image as a python environment%python.docker activate gcr.io/tensorflow/tensorflow:latestUsing Zeppelin Dynamic FormsYou can leverage Zeppelin Dynamic Form inside your Python code.Zeppelin Dynamic Form can only be used if py4j Python library is installed in your system. If not, you can install it with pip install py4j.Example : %python### Input formprint (z.input(&quot;f1&quot;,&quot;defaultValue&quot;))### Select formprint (z.select(&quot;f1&quot;,[(&quot;o1&quot;,&quot;1&quot;),(&quot;o2&quot;,&quot;2&quot;)],&quot;2&quot;))### Checkbox formprint(&quot;&quot;.join(z.checkbox(&quot;f3&quot;, [(&quot;o1&quot;,&quot;1&quot;), (&quot;o2&quot;,&quot;2&quot;)],[&quot;1&quot;]))
)Matplotlib integrationThe python interpreter can display matplotlib figures inline automatically using the pyplot module:%pythonimport matplotlib.pyplot as pltplt.plot([1, 2, 3])This is the recommended method for using matplotlib from within a Zeppelin notebook. The output of this command will by default be converted to HTML by implicitly making use of the %html magic. Additional configuration can be achieved using the builtin z.configure_mpl() method. For example, z.configure_mpl(width=400, height=300, fmt=&#39;svg&#39;)plt.plot([1, 2, 3])Will produce a 400x300 image in SVG format, which by default are normally 600x400 and PNG respectively. In the future, another option called angular can be used to make it possible to update a plot produced from one paragraph directly from another (the output will be %angular instead of %html). However, this feature is already available in the pyspark interpreter. More details can be found in the included &quot;Zeppelin Tutorial: Pyth
on - matplotlib basic&quot; tutorial notebook. If Zeppelin cannot find the matplotlib backend files (which should usually be found in $ZEPPELIN_HOME/interpreter/lib/python) in your PYTHONPATH, then the backend will automatically be set to agg, and the (otherwise deprecated) instructions below can be used for more limited inline plotting.If you are unable to load the inline backend, use z.show(plt): python%pythonimport matplotlib.pyplot as pltplt.figure()(.. ..)z.show(plt)plt.close()The z.show() function can take optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).%pythonz.show(plt, width=&#39;50px&#39;)z.show(plt, height=&#39;150px&#39;, fmt=&#39;svg&#39;)Pandas integrationApache Zeppelin Table Display System provides built-in data visualization capabilities. Python interpreter leverages it to visualize Pandas DataFrames though similar z.show() API, same as with Matplotlib integration.Example:i
mport pandas as pdrates = pd.read_csv(&quot;bank.csv&quot;, sep=&quot;;&quot;)z.show(rates)SQL over Pandas DataFramesThere is a convenience %python.sql interpreter that matches Apache Spark experience in Zeppelin and enables usage of SQL language to query Pandas DataFrames and visualization of results though built-in Table Display System.Pre-requestsPandas pip install pandasPandaSQL pip install -U pandasqlIn case default binded interpreter is Python (first in the interpreter list, under the Gear Icon), you can just use it as %sql i.efirst paragraphimport pandas as pdrates = pd.read_csv(&quot;bank.csv&quot;, sep=&quot;;&quot;)next paragraph%sqlSELECT * FROM rates WHERE age &lt; 40Otherwise it can be referred to as %python.sqlTechnical descriptionFor in-depth technical details on current implementation please refer to python/README.md.Some features not yet implemented in the Python InterpreterInterrupt a paragraph execution (cancel() method) is curr
ently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter. A JIRA ticket (ZEPPELIN-893) is opened to implement this feature in a next release of the interpreter.Progression bar in webUI (getProgress() method) is currently not implemented.Code-completion is currently not implemented.",
"url": " /interpreter/python.html",
"group": "interpreter",
"excerpt": "Python is a programming language that lets you work quickly and integrate systems more effectively."
@@ -402,7 +402,7 @@
"/interpreter/scio.html": {
"title": "Scio Interpreter for Apache Zeppelin",
- "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Scio Interpreter for Apache ZeppelinOverviewScio is a Scala DSL for Google Cloud Dataflow and Apache Beam inspired by Spark and Scalding. See the current wiki and API documentation for more information.Configuration Name Default Value Description zeppelin.scio.argz --runner=InProcessPipelineRunner Scio interpreter wide arguments. Documentation: https://github.com/spotify/scio/wiki#options and https://cloud.g
oogle.com/dataflow/pipelines/specifying-exec-params zeppelin.scio.maxResult 1000 Max number of SCollection results to display Enabling the Scio InterpreterIn a notebook, to enable the Scio interpreter, click the Gear icon and select beam (beam.scio).Using the Scio InterpreterIn a paragraph, use $beam.scio to select the Scio interpreter. You can use it much the same way as vanilla Scala REPL and Scio REPL. State (like variables, imports, execution etc) is shared among all Scio paragraphs. There is a special variable argz which holds arguments from Scio interpreter settings. The easiest way to proceed is to create a Scio context via standard ContextAndArgs.$beam.scioval (sc, args) = ContextAndArgs(argz)Use sc context the way you would in regular pipeline/REPL.Example:$beam.scioval (sc, args) = ContextAndArgs(argz)sc.parallelize(Seq(&quot;foo&quot;, &quot;foo&quot;, &quot;bar&quot;)).countByValue.closeAndDisplay()If you close Scio context, go ahead
an create a new one using ContextAndArgs. Please refer to Scio wiki for more complex examples. You can close Scio context much the same way as in Scio REPL, and use Zeppelin display helpers to synchronously close and display results - read more below.ProgressThere can be only one paragraph running at a time. There is no notion of overall progress, thus progress bar will be 0.SCollection display helpersScio interpreter comes with display helpers to ease working with Zeppelin notebooks. Simply use closeAndDisplay() on SCollection to close context and display the results. The number of results is limited by zeppelin.scio.maxResult (by default 1000).Supported SCollection types:Scio&#39;s typed BigQueryScala&#39;s Products (case classes, tuples)Google BigQuery&#39;s TableRowApache AvroAll Scala&#39;s AnyValHelper methodsThere are different helper methods for different objects. You can easily display results from SCollection, Future[Tap] and Tap.SCollection helperSCollect
ion has closeAndDisplay Zeppelin helper method for types listed above. Use it to synchronously close Scio context, once available pull and display results.Future[Tap] helperFuture[Tap] has waitAndDisplay Zeppelin helper method for types listed above. Use it synchronously wait for results, once available pull and display results.Tap helperTap has display Zeppelin helper method for types listed above. Use it to pull and display results.ExamplesBigQuery example:$beam.scio@BigQueryType.fromQuery(&quot;&quot;&quot;|SELECT departure_airport,count(case when departure_delay&gt;0 then 1 else 0 end) as no_of_delays |FROM [bigquery-samples:airline_ontime_data.flights] |group by departure_airport |order by 2 desc |limit 10&quot;&quot;&quot;.stripMargin) class Flightsval (sc, args) = ContextAndArgs(argz)sc.bigQuerySelect(Flights.query).closeAndDisplay(Flights.schema
)BigQuery typed example:$beam.scio@BigQueryType.fromQuery(&quot;&quot;&quot;|SELECT departure_airport,count(case when departure_delay&gt;0 then 1 else 0 end) as no_of_delays |FROM [bigquery-samples:airline_ontime_data.flights] |group by departure_airport |order by 2 desc |limit 10&quot;&quot;&quot;.stripMargin) class Flightsval (sc, args) = ContextAndArgs(argz)sc.typedBigQuery[Flights]().flatMap(_.no_of_delays).mean.closeAndDisplay()Avro example:$beam.scioimport com.spotify.data.ExampleAvroval (sc, args) = ContextAndArgs(argz)sc.avroFile[ExampleAvro](&quot;gs://&lt;bucket&gt;/tmp/my.avro&quot;).take(10).closeAndDisplay()Avro example with a view schema:$beam.scioimport com.spotify.data.ExampleAvroimport org.apache.avro.Schemaval (sc, args) = ContextAndArgs(argz)val view = Schema.parse(&quot;&quot;&quot;{&quot;type&quot
;:&quot;record&quot;,&quot;name&quot;:&quot;ExampleAvro&quot;,&quot;namespace&quot;:&quot;com.spotify.data&quot;,&quot;fields&quot;:[{&quot;name&quot;:&quot;track&quot;,&quot;type&quot;:&quot;string&quot;}, {&quot;name&quot;:&quot;artist&quot;, &quot;type&quot;:&quot;string&quot;}]}&quot;&quot;&quot;)sc.avroFile[EndSongCleaned](&quot;gs://&lt;bucket&gt;/tmp/my.avro&quot;).take(10).closeAndDisplay(view)Google credentialsScio Interpreter will try to infer your Google Cloud credentials from its environment, it will take into the account: * argz interpreter settings (doc) * environment variable (GOOGLE_APPLICATION_CREDENTIALS) * gcloud configurationBigQuery macro credentialsCurrently BigQuery project for macro expansion is inferred using Google Dataflow&#39;s DefaultProjectFactory().create()",
+ "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Scio Interpreter for Apache ZeppelinOverviewScio is a Scala DSL for Google Cloud Dataflow and Apache Beam inspired by Spark and Scalding. See the current wiki and API documentation for more information.Configuration Name Default Value Description zeppelin.scio.argz --runner=InProcessPipelineRunner Scio interpreter wide arguments. Documentation: https://github.com/spotify/scio/wiki#options and https://cloud.g
oogle.com/dataflow/pipelines/specifying-exec-params zeppelin.scio.maxResult 1000 Max number of SCollection results to display Enabling the Scio InterpreterIn a notebook, to enable the Scio interpreter, click the Gear icon and select beam (beam.scio).Using the Scio InterpreterIn a paragraph, use $beam.scio to select the Scio interpreter. You can use it much the same way as vanilla Scala REPL and Scio REPL. State (like variables, imports, execution etc) is shared among all Scio paragraphs. There is a special variable argz which holds arguments from Scio interpreter settings. The easiest way to proceed is to create a Scio context via standard ContextAndArgs.$beam.scioval (sc, args) = ContextAndArgs(argz)Use sc context the way you would in a regular pipeline/REPL.Example:$beam.scioval (sc, args) = ContextAndArgs(argz)sc.parallelize(Seq(&quot;foo&quot;, &quot;foo&quot;, &quot;bar&quot;)).countByValue.closeAndDisplay()If you close Scio context, go ahe
ad an create a new one using ContextAndArgs. Please refer to Scio wiki for more complex examples. You can close Scio context much the same way as in Scio REPL, and use Zeppelin display helpers to synchronously close and display results - read more below.ProgressThere can be only one paragraph running at once. There is no notion of overall progress, thus progress bar will show 0.SCollection display helpersScio interpreter comes with display helpers to ease working with Zeppelin notebooks. Simply use closeAndDisplay() on SCollection to close context and display the results. The number of results is limited by zeppelin.scio.maxResult (by default 1000).Supported SCollection types:Scio&#39;s typed BigQueryScala&#39;s Products (case classes, tuples)Google BigQuery&#39;s TableRowApache AvroAll Scala&#39;s AnyValHelper methodsThere are different helper methods for different objects. You can easily display results from SCollection, Future[Tap] and Tap.SCollection helperSColle
ction has closeAndDisplay Zeppelin helper method for types listed above. Use it to synchronously close Scio context, and once available pull and display results.Future[Tap] helperFuture[Tap] has waitAndDisplay Zeppelin helper method for types listed above. Use it to synchronously wait for results, and once available pull and display results.Tap helperTap has display Zeppelin helper method for types listed above. Use it to pull and display results.ExamplesBigQuery example:$beam.scio@BigQueryType.fromQuery(&quot;&quot;&quot;|SELECT departure_airport,count(case when departure_delay&gt;0 then 1 else 0 end) as no_of_delays |FROM [bigquery-samples:airline_ontime_data.flights] |group by departure_airport |order by 2 desc |limit 10&quot;&quot;&quot;.stripMargin) class Flightsval (sc, args) = ContextAndArgs(argz)sc.bigQuerySelect(Flights.query).closeAndDisplay(F
lights.schema)BigQuery typed example:$beam.scio@BigQueryType.fromQuery(&quot;&quot;&quot;|SELECT departure_airport,count(case when departure_delay&gt;0 then 1 else 0 end) as no_of_delays |FROM [bigquery-samples:airline_ontime_data.flights] |group by departure_airport |order by 2 desc |limit 10&quot;&quot;&quot;.stripMargin) class Flightsval (sc, args) = ContextAndArgs(argz)sc.typedBigQuery[Flights]().flatMap(_.no_of_delays).mean.closeAndDisplay()Avro example:$beam.scioimport com.spotify.data.ExampleAvroval (sc, args) = ContextAndArgs(argz)sc.avroFile[ExampleAvro](&quot;gs://&lt;bucket&gt;/tmp/my.avro&quot;).take(10).closeAndDisplay()Avro example with a view schema:$beam.scioimport com.spotify.data.ExampleAvroimport org.apache.avro.Schemaval (sc, args) = ContextAndArgs(argz)val view = Schema.parse(&quot;&quot;&quot;{&quot;
type&quot;:&quot;record&quot;,&quot;name&quot;:&quot;ExampleAvro&quot;,&quot;namespace&quot;:&quot;com.spotify.data&quot;,&quot;fields&quot;:[{&quot;name&quot;:&quot;track&quot;,&quot;type&quot;:&quot;string&quot;}, {&quot;name&quot;:&quot;artist&quot;, &quot;type&quot;:&quot;string&quot;}]}&quot;&quot;&quot;)sc.avroFile[EndSongCleaned](&quot;gs://&lt;bucket&gt;/tmp/my.avro&quot;).take(10).closeAndDisplay(view)Google credentialsScio Interpreter will try to infer your Google Cloud credentials from its environment, it will take into the account:argz interpreter settings (doc)environment variable (GOOGLE_APPLICATION_CREDENTIALS)gcloud configurationBigQuery macro credentialsCurrently BigQuery project for macro expansion is inferred using Google Dataflow&#39;s DefaultProjectFactory().create()",
"url": " /interpreter/scio.html",
"group": "interpreter",
"excerpt": "Scio is a Scala DSL for Apache Beam/Google Dataflow model."
@@ -424,7 +424,7 @@
"/interpreter/spark.html": {
"title": "Apache Spark Interpreter for Apache Zeppelin",
[... 51 lines stripped ...]
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/security/authentication.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/security/authentication.html?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/security/authentication.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/security/authentication.html Thu Nov 24 18:26:45 2016
@@ -93,6 +93,7 @@
<li><a href="/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html">Interpreter Installation</a></li>
<!--<li><a href="/docs/0.7.0-SNAPSHOT/manual/dynamicinterpreterload.html">Dynamic Interpreter Loading</a></li>-->
<li><a href="/docs/0.7.0-SNAPSHOT/manual/dependencymanagement.html">Interpreter Dependency Management</a></li>
+ <li><a href="/docs/0.7.0-SNAPSHOT/manual/userimpersonation.html">Interpreter User Impersonation</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Available Interpreters</b><span></li>
<li><a href="/docs/0.7.0-SNAPSHOT/interpreter/alluxio.html">Alluxio</a></li>
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html Thu Nov 24 18:26:45 2016
@@ -93,6 +93,7 @@
<li><a href="/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html">Interpreter Installation</a></li>
<!--<li><a href="/docs/0.7.0-SNAPSHOT/manual/dynamicinterpreterload.html">Dynamic Interpreter Loading</a></li>-->
<li><a href="/docs/0.7.0-SNAPSHOT/manual/dependencymanagement.html">Interpreter Dependency Management</a></li>
+ <li><a href="/docs/0.7.0-SNAPSHOT/manual/userimpersonation.html">Interpreter User Impersonation</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Available Interpreters</b><span></li>
<li><a href="/docs/0.7.0-SNAPSHOT/interpreter/alluxio.html">Alluxio</a></li>
@@ -222,7 +223,10 @@ In this case, you can add your credentia
<p><img class="img-responsive" src="../assets/themes/zeppelin/img/docs-img/credential_tab.png" width="180px"/></p>
-<p><strong>Entity</strong> can be the key that distinguishes each credential sets. Type <strong>Username & Password</strong> for your own credentials. ex) user & password of Mysql </p>
+<p><strong>Entity</strong> can be the key that distinguishes each credential sets.(We suggest that the convention of the <strong>Entity</strong> is <code>[Interpreter Group].[Interpreter Name]</code>.)
+Please see <a href="../manual/interpreters.html#what-is-interpreter-group">what is interpreter group</a> for the detailed information.</p>
+
+<p>Type <strong>Username & Password</strong> for your own credentials. ex) Mysql user & password of the JDBC Interpreter.</p>
<p><img class="img-responsive" src="../assets/themes/zeppelin/img/docs-img/add_credential.png" /></p>
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/security/notebook_authorization.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/security/notebook_authorization.html?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/security/notebook_authorization.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/security/notebook_authorization.html Thu Nov 24 18:26:45 2016
@@ -93,6 +93,7 @@
<li><a href="/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html">Interpreter Installation</a></li>
<!--<li><a href="/docs/0.7.0-SNAPSHOT/manual/dynamicinterpreterload.html">Dynamic Interpreter Loading</a></li>-->
<li><a href="/docs/0.7.0-SNAPSHOT/manual/dependencymanagement.html">Interpreter Dependency Management</a></li>
+ <li><a href="/docs/0.7.0-SNAPSHOT/manual/userimpersonation.html">Interpreter User Impersonation</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Available Interpreters</b><span></li>
<li><a href="/docs/0.7.0-SNAPSHOT/interpreter/alluxio.html">Alluxio</a></li>
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html Thu Nov 24 18:26:45 2016
@@ -93,6 +93,7 @@
<li><a href="/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html">Interpreter Installation</a></li>
<!--<li><a href="/docs/0.7.0-SNAPSHOT/manual/dynamicinterpreterload.html">Dynamic Interpreter Loading</a></li>-->
<li><a href="/docs/0.7.0-SNAPSHOT/manual/dependencymanagement.html">Interpreter Dependency Management</a></li>
+ <li><a href="/docs/0.7.0-SNAPSHOT/manual/userimpersonation.html">Interpreter User Impersonation</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Available Interpreters</b><span></li>
<li><a href="/docs/0.7.0-SNAPSHOT/interpreter/alluxio.html">Alluxio</a></li>
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/sitemap.txt
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/sitemap.txt?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/sitemap.txt (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/sitemap.txt Thu Nov 24 18:26:45 2016
@@ -47,6 +47,7 @@ http://zeppelin.apache.org/manual/dynami
http://zeppelin.apache.org/manual/interpreterinstallation.html
http://zeppelin.apache.org/manual/interpreters.html
http://zeppelin.apache.org/manual/notebookashomepage.html
+http://zeppelin.apache.org/manual/processenduser.html
http://zeppelin.apache.org/manual/publish.html
http://zeppelin.apache.org/pleasecontribute.html
http://zeppelin.apache.org/quickstart/explorezeppelinui.html
Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/storage/storage.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/storage/storage.html?rev=1771200&r1=1771199&r2=1771200&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/storage/storage.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/storage/storage.html Thu Nov 24 18:26:45 2016
@@ -93,6 +93,7 @@
<li><a href="/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html">Interpreter Installation</a></li>
<!--<li><a href="/docs/0.7.0-SNAPSHOT/manual/dynamicinterpreterload.html">Dynamic Interpreter Loading</a></li>-->
<li><a href="/docs/0.7.0-SNAPSHOT/manual/dependencymanagement.html">Interpreter Dependency Management</a></li>
+ <li><a href="/docs/0.7.0-SNAPSHOT/manual/userimpersonation.html">Interpreter User Impersonation</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Available Interpreters</b><span></li>
<li><a href="/docs/0.7.0-SNAPSHOT/interpreter/alluxio.html">Alluxio</a></li>