You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@zeppelin.apache.org by denny wong <te...@yahoo.com.INVALID> on 2021/06/23 14:13:46 UTC

Re: Remote Interpreter

Hi
We are using Zeppelin 0.9 and running the remote interpreter as a docker container.  Currently, the Zeppelin server and the remote interpreter are running on the same machine.  We would like to know if the remote interpreter can be run on a different machine than the Zeppelin server.  Is this currently supported?
Thanks
Denny

Re: Remote Interpreter

Posted by denny wong <te...@yahoo.com.INVALID>.

 Thanks for the quick reply.
In our use case, we only use the python and db interpreters (no yarn, spark, etc).  We would start the Zeppelin server in one node (as a container).  Whenever, user runs a notebook, we would spawn a remote interpreter container (in the same node).  We use the isolated mode (one remote interpreter per note).  If we run out of resources in the current node, we would like to spawn new remote interpreters in a different node (we manage all nodes ourselves).  We are using the DockerInterpreterLauncher to launch the remote interpreter container.
I have a question about this script (used to start the interpreter inside the container).  I see the script accepts the zeppelin.server.rpc.host, which allows the interpreter to communicate back to the zeppelin server. If I start the interpreter in a different node, how does it tell the zeppelin server to connect to that node?  Please advise.
{{CONTAINER_ZEPPELIN_HOME}}/bin/interpreter.sh -d {{CONTAINER_ZEPPELIN_HOME}}/interpreter/{{zeppelin.interpreter.group.name}} -r {{zeppelin.interpreter.rpc.portRange}} -c {{zeppelin.server.rpc.host}} -p {{zeppelin.server.rpc.portRange}} -i {{zeppelin.interpreter.group.id}} -l {{zeppelin.interpreter.localRepo}} -g {{zeppelin.interpreter.setting.name}}

    On Wednesday, June 23, 2021, 11:17:00 AM EDT, Jeff Zhang <zj...@gmail.com> wrote:  

 Yes，it is supported. You can either k8s mode (interpreter runs in pod ) or
yarn mode (interpreter runs in yarn container)

http://zeppelin.apache.org/docs/0.9.0/quickstart/kubernetes.html
http://zeppelin.apache.org/docs/0.9.0/quickstart/yarn.html

denny wong <te...@yahoo.com.invalid> 于2021年6月23日周三 下午10:14写道：

> Hi
> We are using Zeppelin 0.9 and running the remote interpreter as a docker
> container.  Currently, the Zeppelin server and the remote interpreter are
> running on the same machine.  We would like to know if the remote
> interpreter can be run on a different machine than the Zeppelin server.  Is
> this currently supported?
> Thanks
> Denny

-- 
Best Regards

Jeff Zhang

Re: Remote Interpreter

Posted by Jeff Zhang <zj...@gmail.com>.

Yes，it is supported. You can either k8s mode (interpreter runs in pod ) or
yarn mode (interpreter runs in yarn container)

http://zeppelin.apache.org/docs/0.9.0/quickstart/kubernetes.html
http://zeppelin.apache.org/docs/0.9.0/quickstart/yarn.html


denny wong <te...@yahoo.com.invalid> 于2021年6月23日周三 下午10:14写道：

> Hi
> We are using Zeppelin 0.9 and running the remote interpreter as a docker
> container.  Currently, the Zeppelin server and the remote interpreter are
> running on the same machine.  We would like to know if the remote
> interpreter can be run on a different machine than the Zeppelin server.  Is
> this currently supported?
> Thanks
> Denny



-- 
Best Regards

Jeff Zhang