You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by denny wong <te...@yahoo.com.INVALID> on 2018/08/02 04:02:12 UTC
Re: Python Container
Hi
I tried to use the python docker option in 0.8 (see below), but it didn't work for me. I can see the python docker image downloaded properly. However when I ran the python paragraph (2nd paragraph), I got the "process not running" error. I didn't see any python container running. What's wrong? Any idea? - Thanks, Denny
Re: R Container
Posted by te...@yahoo.com.INVALID.
I am interested in R interpreter docker support similar to Python docker support in 0.8. Is there any plan to enable R container support?
Thanks
Denny
Re: Python Container
Posted by denny wong <te...@yahoo.com.INVALID>.
I am still trying to make the python docker to work.
I can see in the debugger, it tries to execute the following docker command to start the python process.
docker run -i --rm -v /tmp:/_zeppelin_tmp -v /scratch/zeppelin-branch-0.8:/_zeppelin -e PYTHONPATH=":/_zeppelin/interpreter/python/py4j-0.10.7/src::/_zeppelin/interpreter/lib/python" python:3.7 python /_zeppelin_tmp/zeppelin_python-8643449992792254752.py 26068 10.196.10.37Traceback (most recent call last): File "/_zeppelin_tmp/zeppelin_python-8643449992792254752.py", line 20, in <module> from py4j.java_gateway import java_import, JavaGateway, GatewayClientModuleNotFoundError: No module named 'py4j'
I see there is a reference to this directory (in -e PYTHONPATH) in the docker run command, but there is no such directory.
/_zeppelin/interpreter/python/py4j-0.10.7/src
However, there is a /_zeppelin/interpreter/python/py4j-0.10.7.jar, but I don't see any "src" subdirectory inside.
What is this "src" suppose to reference? How is the "py4j" module installed? By who?
Please advise!
Denny
On Saturday, August 4, 2018, 10:43:02 PM EDT, Jeff Zhang <zj...@gmail.com> wrote:
impersonation is not supported for ipython. The workaround is to use
ipyspark because spark interpreter support impersonation.
denny wong <te...@yahoo.com.invalid>于2018年8月5日周日 上午10:35写道：
> I wonder if IPython can support multi-user? I need os level isolation
> for python notebook. For example, note1 and note2 shouldn't see each other
> directory files or os environment variables.
> Thanks
> Denny
> On Saturday, August 4, 2018, 9:49:02 PM EDT, Jeff Zhang <
> zjffdu@gmail.com> wrote:
>
> Does you try the link here https://zeppelin.apache.org/download.html ? It
> works for me
>
> Regarding the ipython, the ipython session won't get closed after notebook
> is closed. You can reuse the session after the note is reopened. You check
> the section of Interprter Lifecycle Manager of this blog
> https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235
> Also it includes one ipython interpreter tutorial.
>
> -
>
>
>
> denny wong <te...@yahoo.com.invalid>于2018年8月5日周日 上午12:21写道：
>
> > Hi
> >
> > I tried to download the 0.8 source codes, but the link didn't work. How
> do
> > I get the 0.8 source code?
> >
> >
> > - Source: zeppelin-0.8.0.tgz (58 MB, pgp
> > <
> https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.asc
> >
> > , md5
> > <
> https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.md5
> >
> > , sha
> > <
> https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.sha512
> >
> > )
> >
> >
> > I have been trying to understand the python container behavior, but so
> far
> > I couldn't make it to work. The python.docker works to the point it
> > downloads the python image. The python code execution didn't work in the
> > next paragraph.
> >
> > I want to know about the lifetime of the container, basically the scope
> of
> > the container. If I close the notebook, what happens to the python
> objects
> > (e.g. variables, data frame, model, etc) created, are they gone? For
> > example, the next time I open the same notebook and try to reference the
> > objects the python codes produced previously, am I able to reference them
> > or they are gone?
> >
> > How about the IPython implementation, what is scope of the python
> > session? Is it tied to the notebook session? In the other words, when
> the
> > notebook is closed, all python objects created in the session are gone?
> Is
> > there any way to save the python session so that the last created python
> > objects can be referenced later?
> >
> > Thanks,
> > Denny
> >
> >
> > On Thursday, August 2, 2018, 12:02:12 AM EDT, denny wong <
> > tecgie88@yahoo.com> wrote:
> >
> >
> > Hi
> >
> > I tried to use the python docker option in 0.8 (see below), but it didn't
> > work for me. I can see the python docker image downloaded properly.
> > However when I ran the python paragraph (2nd paragraph), I got the
> "process
> > not running" error. I didn't see any python container running. What's
> > wrong? Any idea? - Thanks, Denny
> >
> >
> > [image: Inline image]
> >
> >
> > [image: Inline image]
> >
> >
Re: Python Container
Posted by Jeff Zhang <zj...@gmail.com>.
impersonation is not supported for ipython. The workaround is to use
ipyspark because spark interpreter support impersonation.
denny wong <te...@yahoo.com.invalid>于2018年8月5日周日 上午10:35写道：
> I wonder if IPython can support multi-user? I need os level isolation
> for python notebook. For example, note1 and note2 shouldn't see each other
> directory files or os environment variables.
> Thanks
> Denny
> On Saturday, August 4, 2018, 9:49:02 PM EDT, Jeff Zhang <
> zjffdu@gmail.com> wrote:
>
> Does you try the link here https://zeppelin.apache.org/download.html ? It
> works for me
>
> Regarding the ipython, the ipython session won't get closed after notebook
> is closed. You can reuse the session after the note is reopened. You check
> the section of Interprter Lifecycle Manager of this blog
> https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235
> Also it includes one ipython interpreter tutorial.
>
> -
>
>
>
> denny wong <te...@yahoo.com.invalid>于2018年8月5日周日 上午12:21写道：
>
> > Hi
> >
> > I tried to download the 0.8 source codes, but the link didn't work. How
> do
> > I get the 0.8 source code?
> >
> >
> > - Source: zeppelin-0.8.0.tgz (58 MB, pgp
> > <
> https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.asc
> >
> > , md5
> > <
> https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.md5
> >
> > , sha
> > <
> https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.sha512
> >
> > )
> >
> >
> > I have been trying to understand the python container behavior, but so
> far
> > I couldn't make it to work. The python.docker works to the point it
> > downloads the python image. The python code execution didn't work in the
> > next paragraph.
> >
> > I want to know about the lifetime of the container, basically the scope
> of
> > the container. If I close the notebook, what happens to the python
> objects
> > (e.g. variables, data frame, model, etc) created, are they gone? For
> > example, the next time I open the same notebook and try to reference the
> > objects the python codes produced previously, am I able to reference them
> > or they are gone?
> >
> > How about the IPython implementation, what is scope of the python
> > session? Is it tied to the notebook session? In the other words, when
> the
> > notebook is closed, all python objects created in the session are gone?
> Is
> > there any way to save the python session so that the last created python
> > objects can be referenced later?
> >
> > Thanks,
> > Denny
> >
> >
> > On Thursday, August 2, 2018, 12:02:12 AM EDT, denny wong <
> > tecgie88@yahoo.com> wrote:
> >
> >
> > Hi
> >
> > I tried to use the python docker option in 0.8 (see below), but it didn't
> > work for me. I can see the python docker image downloaded properly.
> > However when I ran the python paragraph (2nd paragraph), I got the
> "process
> > not running" error. I didn't see any python container running. What's
> > wrong? Any idea? - Thanks, Denny
> >
> >
> > [image: Inline image]
> >
> >
> > [image: Inline image]
> >
> >
Re: Python Container
Posted by denny wong <te...@yahoo.com.INVALID>.
I wonder if IPython can support multi-user? I need os level isolation for python notebook. For example, note1 and note2 shouldn't see each other directory files or os environment variables.
Thanks
Denny
On Saturday, August 4, 2018, 9:49:02 PM EDT, Jeff Zhang <zj...@gmail.com> wrote:
Does you try the link here https://zeppelin.apache.org/download.html ? It
works for me
Regarding the ipython, the ipython session won't get closed after notebook
is closed. You can reuse the session after the note is reopened. You check
the section of Interprter Lifecycle Manager of this blog
https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235
Also it includes one ipython interpreter tutorial.
-
denny wong <te...@yahoo.com.invalid>于2018年8月5日周日 上午12:21写道：
> Hi
>
> I tried to download the 0.8 source codes, but the link didn't work. How do
> I get the 0.8 source code?
>
>
> - Source: zeppelin-0.8.0.tgz (58 MB, pgp
> <https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.asc>
> , md5
> <https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.md5>
> , sha
> <https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.sha512>
> )
>
>
> I have been trying to understand the python container behavior, but so far
> I couldn't make it to work. The python.docker works to the point it
> downloads the python image. The python code execution didn't work in the
> next paragraph.
>
> I want to know about the lifetime of the container, basically the scope of
> the container. If I close the notebook, what happens to the python objects
> (e.g. variables, data frame, model, etc) created, are they gone? For
> example, the next time I open the same notebook and try to reference the
> objects the python codes produced previously, am I able to reference them
> or they are gone?
>
> How about the IPython implementation, what is scope of the python
> session? Is it tied to the notebook session? In the other words, when the
> notebook is closed, all python objects created in the session are gone? Is
> there any way to save the python session so that the last created python
> objects can be referenced later?
>
> Thanks,
> Denny
>
>
> On Thursday, August 2, 2018, 12:02:12 AM EDT, denny wong <
> tecgie88@yahoo.com> wrote:
>
>
> Hi
>
> I tried to use the python docker option in 0.8 (see below), but it didn't
> work for me. I can see the python docker image downloaded properly.
> However when I ran the python paragraph (2nd paragraph), I got the "process
> not running" error. I didn't see any python container running. What's
> wrong? Any idea? - Thanks, Denny
>
>
> [image: Inline image]
>
>
> [image: Inline image]
>
>
Re: Python Container
Posted by Jeff Zhang <zj...@gmail.com>.
Does you try the link here https://zeppelin.apache.org/download.html ? It
works for me
Regarding the ipython, the ipython session won't get closed after notebook
is closed. You can reuse the session after the note is reopened. You check
the section of Interprter Lifecycle Manager of this blog
https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235
Also it includes one ipython interpreter tutorial.
-
denny wong <te...@yahoo.com.invalid>于2018年8月5日周日 上午12:21写道：
> Hi
>
> I tried to download the 0.8 source codes, but the link didn't work. How do
> I get the 0.8 source code?
>
>
> - Source: zeppelin-0.8.0.tgz (58 MB, pgp
> <https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.asc>
> , md5
> <https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.md5>
> , sha
> <https://www.apache.org/dist/zeppelin/zeppelin-0.8.0/zeppelin-0.8.0.tgz.sha512>
> )
>
>
> I have been trying to understand the python container behavior, but so far
> I couldn't make it to work. The python.docker works to the point it
> downloads the python image. The python code execution didn't work in the
> next paragraph.
>
> I want to know about the lifetime of the container, basically the scope of
> the container. If I close the notebook, what happens to the python objects
> (e.g. variables, data frame, model, etc) created, are they gone? For
> example, the next time I open the same notebook and try to reference the
> objects the python codes produced previously, am I able to reference them
> or they are gone?
>
> How about the IPython implementation, what is scope of the python
> session? Is it tied to the notebook session? In the other words, when the
> notebook is closed, all python objects created in the session are gone? Is
> there any way to save the python session so that the last created python
> objects can be referenced later?
>
> Thanks,
> Denny
>
>
> On Thursday, August 2, 2018, 12:02:12 AM EDT, denny wong <
> tecgie88@yahoo.com> wrote:
>
>
> Hi
>
> I tried to use the python docker option in 0.8 (see below), but it didn't
> work for me. I can see the python docker image downloaded properly.
> However when I ran the python paragraph (2nd paragraph), I got the "process
> not running" error. I didn't see any python container running. What's
> wrong? Any idea? - Thanks, Denny
>
>
> [image: Inline image]
>
>
> [image: Inline image]
>
>
Re: Python Container
Posted by denny wong <te...@yahoo.com.INVALID>.
Hi
I tried to download the 0.8 source codes, but the link didn't work. How do I get the 0.8 source code?
- Source: zeppelin-0.8.0.tgz (58 MB, pgp, md5, sha)
I have been trying to understand the python container behavior, but so far I couldn't make it to work. The python.docker works to the point it downloads the python image. The python code execution didn't work in the next paragraph.
I want to know about the lifetime of the container, basically the scope of the container. If I close the notebook, what happens to the python objects (e.g. variables, data frame, model, etc) created, are they gone? For example, the next time I open the same notebook and try to reference the objects the python codes produced previously, am I able to reference them or they are gone?
How about the IPython implementation, what is scope of the python session? Is it tied to the notebook session? In the other words, when the notebook is closed, all python objects created in the session are gone? Is there any way to save the python session so that the last created python objects can be referenced later?
Thanks,
Denny
On Thursday, August 2, 2018, 12:02:12 AM EDT, denny wong <te...@yahoo.com> wrote:
Hi
I tried to use the python docker option in 0.8 (see below), but it didn't work for me. I can see the python docker image downloaded properly. However when I ran the python paragraph (2nd paragraph), I got the "process not running" error. I didn't see any python container running. What's wrong? Any idea? - Thanks, Denny