You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Igor Yakushin <iv...@uchicago.edu> on 2016/11/06 03:30:44 UTC
under what userid should Zeppelin run?
Hi All,
Under what userid should Zeppelin run? The documentation does not seem
to mention it.
Thank you,
Igor
Re: under what userid should Zeppelin run?
Posted by Jeff Zhang <zj...@gmail.com>.
I think you are referring impersonation. It is still in progress.
Here's one related ticket
https://issues.apache.org/jira/browse/ZEPPELIN-1320
moon soo Lee <mo...@apache.org>于2016年11月7日周一 上午10:48写道：
> Zeppelin already have three different modes for interpreter process
> management.
>
> - shared : share a single process per note
> - scoped: share a single process but create instance inside per note
> - isolated : create a new process per note.
>
> Now community also is actively working on bring this ability to not only
> per note but also per user as well. Here's related issues. And i think
> we'll have those features really soon.
> https://issues.apache.org/jira/browse/ZEPPELIN-1210
> https://issues.apache.org/jira/browse/ZEPPELIN-1236
>
> Let me know if you have any other questions.
>
> Thanks,
> moon
>
> On Sun, Nov 6, 2016 at 5:45 PM Igor Yakushin <iv...@uchicago.edu> wrote:
>
>
>
> On 11/06/2016 07:30 PM, moon soo Lee wrote:
> > Hi Igor,
> >
> > Zeppelin runs with user id that execute bin/zeppelin-daemon.sh or
> > bin/zeppeiln.sh. And all interpreter processes are running with the
> > same user id at the moment. So you'll need to change your user id to
> > appropriate one for your job and start Zeppelin.
>
> Hi Moon,
> So do I understand correctly that at the moment one cannot run a single
> Zeppelin server for all the users to connect to and each user is
> supposed to run his own instance of the server?
> Zeppelin can use ldap for authentication so I thought that I can run it
> as root. But later I discovered that %sh would give root access to all
> the users!!!
> Is there a way around it? Like running it under some service account?
> With Jupyter, for example, there is JupyterHub that allows that: each
> user only has access to his account. Nothing like that for Zeppelin?
> Thank you,
> Igor
>
> --
> Igor Yakushin, Ph.D.
> Computational Scientist
> Kavli Institute for Cosmological Physics, ERC #413
> Research Computing Center, room #2
> The University of Chicago
> 773-834-5313 <(773)%20834-5313>
> https://rcc.uchicago.edu
>
>
Re: under what userid should Zeppelin run?
Posted by moon soo Lee <mo...@apache.org>.
Interpreters can be restarted from 'interpreter' menu, or individual user
can restart interpreter in 'interpreter binding' menu on each Notes. Second
method available in 0.7.0-SNAPSHOT.
'Job' menu (in 0.7.0-SNAPSHOT) supposed to provide at a glance view of all
jobs running on Zeppelin and will help admin manage the resources.
Hope this helps.
Thanks,
moon
On Mon, Nov 7, 2016 at 3:21 PM York Huang <yo...@gmail.com> wrote:
> Thanks moon.
>
> One more question may not be so related. How can user shut down individual
> notebook and its interpreter? How does the administrator monitor the
> running interpreter and shut it down if running out of resources?
>
> On 8 November 2016 at 05:33, moon soo Lee <mo...@apache.org> wrote:
>
> Thanks Igor for valuable feedbacks.
> For that reason, i've seen some companies instantiate Zeppelin inside of
> container per person, or restrict usage of certain type of interpreters.
> I expect Zeppelin will have impersonation soon and mitigate that problem
> from the next release.
>
> York,
> Yes, you can run multiple Zeppelin instances with different configurations.
> bin/zeppelin-daemon.sh accepts --config parameter. You can have multiple
> configuration directories and point them with --config parameter.
>
> Thanks,
> moon
>
> On Sun, Nov 6, 2016 at 10:39 PM York Huang <yo...@gmail.com>
> wrote:
>
> Also if I want to run all Zeppelin instances from one Linux machine, do I
> have to create the Zeppelin binaries for each user with different ports so
> that multiple instances can run in the same box?
>
> Sent from my iPhone
>
> On 7 Nov 2016, at 5:17 PM, Igor Yakushin <iv...@uchicago.edu> wrote:
>
> Upon reading the documentation and coming with JupyterHub experience, it
> was natural to assume that Zeppelin should run as root, one daemon for all
> the users, especially considering that one can use it with ldap.
>
> One should put it into the documentation with big red letters:
> *NEVER RUN ZEPPELIN AS ROOT !!! *
>
> Also, the daemon itself should check if it is running as root and die with
> violent complaints if yes. This is a major security problem.
>
> This was not obvious at all to me that each user should run his own daemon
> on his account. I know at least one more place that made the same mistake
> as me and runs Zeppelin under root.
>
>
>
Re: under what userid should Zeppelin run?
Posted by York Huang <yo...@gmail.com>.
Thanks moon.
One more question may not be so related. How can user shut down individual
notebook and its interpreter? How does the administrator monitor the
running interpreter and shut it down if running out of resources?
On 8 November 2016 at 05:33, moon soo Lee <mo...@apache.org> wrote:
> Thanks Igor for valuable feedbacks.
> For that reason, i've seen some companies instantiate Zeppelin inside of
> container per person, or restrict usage of certain type of interpreters.
> I expect Zeppelin will have impersonation soon and mitigate that problem
> from the next release.
>
> York,
> Yes, you can run multiple Zeppelin instances with different configurations.
> bin/zeppelin-daemon.sh accepts --config parameter. You can have multiple
> configuration directories and point them with --config parameter.
>
> Thanks,
> moon
>
> On Sun, Nov 6, 2016 at 10:39 PM York Huang <yo...@gmail.com>
> wrote:
>
>> Also if I want to run all Zeppelin instances from one Linux machine, do I
>> have to create the Zeppelin binaries for each user with different ports so
>> that multiple instances can run in the same box?
>>
>> Sent from my iPhone
>>
>> On 7 Nov 2016, at 5:17 PM, Igor Yakushin <iv...@uchicago.edu> wrote:
>>
>> Upon reading the documentation and coming with JupyterHub experience, it
>> was natural to assume that Zeppelin should run as root, one daemon for all
>> the users, especially considering that one can use it with ldap.
>>
>> One should put it into the documentation with big red letters:
>> *NEVER RUN ZEPPELIN AS ROOT !!! *
>>
>> Also, the daemon itself should check if it is running as root and die
>> with violent complaints if yes. This is a major security problem.
>>
>> This was not obvious at all to me that each user should run his own
>> daemon on his account. I know at least one more place that made the same
>> mistake as me and runs Zeppelin under root.
>>
>>
Re: under what userid should Zeppelin run?
Posted by moon soo Lee <mo...@apache.org>.
Thanks Igor for valuable feedbacks.
For that reason, i've seen some companies instantiate Zeppelin inside of
container per person, or restrict usage of certain type of interpreters.
I expect Zeppelin will have impersonation soon and mitigate that problem
from the next release.
York,
Yes, you can run multiple Zeppelin instances with different configurations.
bin/zeppelin-daemon.sh accepts --config parameter. You can have multiple
configuration directories and point them with --config parameter.
Thanks,
moon
On Sun, Nov 6, 2016 at 10:39 PM York Huang <yo...@gmail.com> wrote:
> Also if I want to run all Zeppelin instances from one Linux machine, do I
> have to create the Zeppelin binaries for each user with different ports so
> that multiple instances can run in the same box?
>
> Sent from my iPhone
>
> On 7 Nov 2016, at 5:17 PM, Igor Yakushin <iv...@uchicago.edu> wrote:
>
> Upon reading the documentation and coming with JupyterHub experience, it
> was natural to assume that Zeppelin should run as root, one daemon for all
> the users, especially considering that one can use it with ldap.
>
> One should put it into the documentation with big red letters:
> *NEVER RUN ZEPPELIN AS ROOT !!! *
>
> Also, the daemon itself should check if it is running as root and die with
> violent complaints if yes. This is a major security problem.
>
> This was not obvious at all to me that each user should run his own daemon
> on his account. I know at least one more place that made the same mistake
> as me and runs Zeppelin under root.
>
>
Re: under what userid should Zeppelin run?
Posted by York Huang <yo...@gmail.com>.
Also if I want to run all Zeppelin instances from one Linux machine, do I have to create the Zeppelin binaries for each user with different ports so that multiple instances can run in the same box?
Sent from my iPhone
> On 7 Nov 2016, at 5:17 PM, Igor Yakushin <iv...@uchicago.edu> wrote:
>
> Upon reading the documentation and coming with JupyterHub experience, it was natural to assume that Zeppelin should run as root, one daemon for all the users, especially considering that one can use it with ldap.
> One should put it into the documentation with big red letters: NEVER RUN ZEPPELIN AS ROOT !!!
>
> Also, the daemon itself should check if it is running as root and die with violent complaints if yes. This is a major security problem.
>
> This was not obvious at all to me that each user should run his own daemon on his account. I know at least one more place that made the same mistake as me and runs Zeppelin under root.
>
>
Re: under what userid should Zeppelin run?
Posted by Igor Yakushin <iv...@uchicago.edu>.
Upon reading the documentation and coming with JupyterHub experience, it
was natural to assume that Zeppelin should run as root, one daemon for
all the users, especially considering that one can use it with ldap.
One should put it into the documentation with big red letters: *NEVER
RUN ZEPPELIN AS ROOT !!!
*
Also, the daemon itself should check if it is running as root and die
with violent complaints if yes. This is a major security problem.
This was not obvious at all to me that each user should run his own
daemon on his account. I know at least one more place that made the same
mistake as me and runs Zeppelin under root.
Re: under what userid should Zeppelin run?
Posted by moon soo Lee <mo...@apache.org>.
Hi,
That's right. Until Zeppelin get impersonation implemented that i mentioned
in one of previous email.
Thanks Igor for sharing your interests.
Regards,
moon
On Sun, Nov 6, 2016 at 8:42 PM Igor Yakushin <iv...@uchicago.edu> wrote:
> Hi Moon,
>
> This allows a user to specify who can see his note, right?
>
> But that does not protect the system from users getting root access via
> %sh (or who knows what else) if Zeppelin is running as root?
>
> Thank you,
>
> Igor
>
> On 11/06/2016 08:48 PM, moon soo Lee wrote:
>
> Zeppelin already have three different modes for interpreter process
> management.
>
> - shared : share a single process per note
> - scoped: share a single process but create instance inside per note
> - isolated : create a new process per note.
>
> Now community also is actively working on bring this ability to not only
> per note but also per user as well. Here's related issues. And i think
> we'll have those features really soon.
> https://issues.apache.org/jira/browse/ZEPPELIN-1210
> https://issues.apache.org/jira/browse/ZEPPELIN-1236
>
> Let me know if you have any other questions.
>
> Thanks,
> moon
>
> On Sun, Nov 6, 2016 at 5:45 PM Igor Yakushin <iv...@uchicago.edu> wrote:
>
>
>
> On 11/06/2016 07:30 PM, moon soo Lee wrote:
> > Hi Igor,
> >
> > Zeppelin runs with user id that execute bin/zeppelin-daemon.sh or
> > bin/zeppeiln.sh. And all interpreter processes are running with the
> > same user id at the moment. So you'll need to change your user id to
> > appropriate one for your job and start Zeppelin.
>
> Hi Moon,
> So do I understand correctly that at the moment one cannot run a single
> Zeppelin server for all the users to connect to and each user is
> supposed to run his own instance of the server?
> Zeppelin can use ldap for authentication so I thought that I can run it
> as root. But later I discovered that %sh would give root access to all
> the users!!!
> Is there a way around it? Like running it under some service account?
> With Jupyter, for example, there is JupyterHub that allows that: each
> user only has access to his account. Nothing like that for Zeppelin?
> Thank you,
> Igor
>
> --
> Igor Yakushin, Ph.D.
> Computational Scientist
> Kavli Institute for Cosmological Physics, ERC #413
> Research Computing Center, room #2
> The University of Chicago
> 773-834-5313 <%28773%29%20834-5313>
> https://rcc.uchicago.edu
>
>
> --
> Igor Yakushin, Ph.D.
> Computational Scientist
> Kavli Institute for Cosmological Physics, ERC #413
> Research Computing Center, room #2
> The University of Chicago773-834-5313 <(773)%20834-5313>https://rcc.uchicago.edu
>
>
Re: under what userid should Zeppelin run?
Posted by Igor Yakushin <iv...@uchicago.edu>.
Hi Moon,
This allows a user to specify who can see his note, right?
But that does not protect the system from users getting root access via
%sh (or who knows what else) if Zeppelin is running as root?
Thank you,
Igor
On 11/06/2016 08:48 PM, moon soo Lee wrote:
> Zeppelin already have three different modes for interpreter process
> management.
>
> - shared : share a single process per note
> - scoped: share a single process but create instance inside per note
> - isolated : create a new process per note.
>
> Now community also is actively working on bring this ability to not
> only per note but also per user as well. Here's related issues. And i
> think we'll have those features really soon.
> https://issues.apache.org/jira/browse/ZEPPELIN-1210
> https://issues.apache.org/jira/browse/ZEPPELIN-1236
>
> Let me know if you have any other questions.
>
> Thanks,
> moon
>
> On Sun, Nov 6, 2016 at 5:45 PM Igor Yakushin <ivy2@uchicago.edu
> <ma...@uchicago.edu>> wrote:
>
>
>
> On 11/06/2016 07:30 PM, moon soo Lee wrote:
> > Hi Igor,
> >
> > Zeppelin runs with user id that execute bin/zeppelin-daemon.sh or
> > bin/zeppeiln.sh. And all interpreter processes are running with the
> > same user id at the moment. So you'll need to change your user id to
> > appropriate one for your job and start Zeppelin.
>
> Hi Moon,
> So do I understand correctly that at the moment one cannot run a
> single
> Zeppelin server for all the users to connect to and each user is
> supposed to run his own instance of the server?
> Zeppelin can use ldap for authentication so I thought that I can
> run it
> as root. But later I discovered that %sh would give root access to all
> the users!!!
> Is there a way around it? Like running it under some service account?
> With Jupyter, for example, there is JupyterHub that allows that: each
> user only has access to his account. Nothing like that for Zeppelin?
> Thank you,
> Igor
>
> --
> Igor Yakushin, Ph.D.
> Computational Scientist
> Kavli Institute for Cosmological Physics, ERC #413
> Research Computing Center, room #2
> The University of Chicago
> 773-834-5313 <tel:%28773%29%20834-5313>
> https://rcc.uchicago.edu
>
--
Igor Yakushin, Ph.D.
Computational Scientist
Kavli Institute for Cosmological Physics, ERC #413
Research Computing Center, room #2
The University of Chicago
773-834-5313
https://rcc.uchicago.edu
Re: under what userid should Zeppelin run?
Posted by moon soo Lee <mo...@apache.org>.
Zeppelin already have three different modes for interpreter process
management.
- shared : share a single process per note
- scoped: share a single process but create instance inside per note
- isolated : create a new process per note.
Now community also is actively working on bring this ability to not only
per note but also per user as well. Here's related issues. And i think
we'll have those features really soon.
https://issues.apache.org/jira/browse/ZEPPELIN-1210
https://issues.apache.org/jira/browse/ZEPPELIN-1236
Let me know if you have any other questions.
Thanks,
moon
On Sun, Nov 6, 2016 at 5:45 PM Igor Yakushin <iv...@uchicago.edu> wrote:
>
>
> On 11/06/2016 07:30 PM, moon soo Lee wrote:
> > Hi Igor,
> >
> > Zeppelin runs with user id that execute bin/zeppelin-daemon.sh or
> > bin/zeppeiln.sh. And all interpreter processes are running with the
> > same user id at the moment. So you'll need to change your user id to
> > appropriate one for your job and start Zeppelin.
>
> Hi Moon,
> So do I understand correctly that at the moment one cannot run a single
> Zeppelin server for all the users to connect to and each user is
> supposed to run his own instance of the server?
> Zeppelin can use ldap for authentication so I thought that I can run it
> as root. But later I discovered that %sh would give root access to all
> the users!!!
> Is there a way around it? Like running it under some service account?
> With Jupyter, for example, there is JupyterHub that allows that: each
> user only has access to his account. Nothing like that for Zeppelin?
> Thank you,
> Igor
>
> --
> Igor Yakushin, Ph.D.
> Computational Scientist
> Kavli Institute for Cosmological Physics, ERC #413
> Research Computing Center, room #2
> The University of Chicago
> 773-834-5313 <(773)%20834-5313>
> https://rcc.uchicago.edu
>
>
Re: under what userid should Zeppelin run?
Posted by Igor Yakushin <iv...@uchicago.edu>.
On 11/06/2016 07:30 PM, moon soo Lee wrote:
> Hi Igor,
>
> Zeppelin runs with user id that execute bin/zeppelin-daemon.sh or
> bin/zeppeiln.sh. And all interpreter processes are running with the
> same user id at the moment. So you'll need to change your user id to
> appropriate one for your job and start Zeppelin.
Hi Moon,
So do I understand correctly that at the moment one cannot run a single
Zeppelin server for all the users to connect to and each user is
supposed to run his own instance of the server?
Zeppelin can use ldap for authentication so I thought that I can run it
as root. But later I discovered that %sh would give root access to all
the users!!!
Is there a way around it? Like running it under some service account?
With Jupyter, for example, there is JupyterHub that allows that: each
user only has access to his account. Nothing like that for Zeppelin?
Thank you,
Igor
--
Igor Yakushin, Ph.D.
Computational Scientist
Kavli Institute for Cosmological Physics, ERC #413
Research Computing Center, room #2
The University of Chicago
773-834-5313
https://rcc.uchicago.edu
Re: under what userid should Zeppelin run?
Posted by moon soo Lee <mo...@apache.org>.
Hi Igor,
Zeppelin runs with user id that execute bin/zeppelin-daemon.sh or
bin/zeppeiln.sh. And all interpreter processes are running with the same
user id at the moment. So you'll need to change your user id to appropriate
one for your job and start Zeppelin.
In case of you're interested in impersonation, community is working on it
and you can track related issues
https://issues.apache.org/jira/browse/ZEPPELIN-1320
https://issues.apache.org/jira/browse/ZEPPELIN-1587
https://issues.apache.org/jira/browse/ZEPPELIN-1567
Hope this helps.
Thanks,
moon
On Sat, Nov 5, 2016 at 8:30 PM Igor Yakushin <iv...@uchicago.edu> wrote:
> Hi All,
> Under what userid should Zeppelin run? The documentation does not seem
> to mention it.
> Thank you,
> Igor
>