You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by kebarcla <kb...@ancestry.com> on 2017/05/27 00:58:03 UTC
Zeppelin 0.7.1,config file not being read
Hello,
I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.
In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.
After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?
Thanks for any pointers,
Ken
# shiro.ini changes:
< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Re: Zeppelin 0.7.1,config file not being read
Posted by Ken Barclay <kb...@ancestry.com>.
You seem like right person to talk to Jonathan!
Something as close as possible to SSO for users of our EMR cluster who use Spark & Hive is what we’re seeking. EMR is so powerful & easy to configure that creating a cluster of any size and type is trivial. Now we need to be provide data scientists a means to use it, and notebook environments like Jupyter and Zeppelin seem ideal. So this is what we’re trying first.
I initially tried to use Hue as my interface, and successfully integrated it with LDAP so that I could authenticate users and groups in the UI – beautiful. And I can navigate to an S3 bucket from Hue where I have Zep notebooks stored, but couldn’t figure out how to then bring them up in Zeppelin. If I could do that my problem would be solved, because I could have users log in as themselves, then bring up their notebooks from a common bucket.
Next thought was to forget about LDAP and just see if I could configure some users in Zep directly via shiro. To that end I logged onto the master node and added this to shiro.ini:
/api/version = authc,roles[admin]
/api/interpreter/** = authc, roles[admin]
/api/configurations/** = authc, roles[admin]
/api/credential/** = authc, roles[admin]
Then created a local user for myself with adduser and put this in zeppelin-env.sh:
export ZEPPELIN_NOTEBOOK_S3_BUCKET=zeppelin
export ZEPPELIN_NOTEBOOK_S3_USER=kbarclay
export ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo
export ZEPPELIN_NOTEBOOK_PUBLIC=true
and I changed one property in zeppelin-site.xml, because according to https://zeppelin.apache.org/docs/0.7.1/install/configuration.html, it cannot be changed in zeppelin-env.sh:
<property>
<name>zeppelin.anonymous.allowed</name>
<value>false</value>
<description>Anonymous user allowed by default</description>
</property>
But after restarting Zeppelin server, it automatically makes me user ‘anonymous’ still, and offers no means of logging out and then back in as another user.
Interestingly, if I put in one of our real bucket names for ZEPPELIN_NOTEBOOK_S3_BUCKET, I get an error in the log file that says it can’t read it and that it’s ignoring the config file and using defaults. So it *is* reading the config file. If I set it back to the default (zeppelin), I don’t get a message about an invalid config file, but I also don’t see any of my configuration changes in the Configuration pull-down (like ZEPPELIN_NOTEBOOK_STORAGE) – everything is set to defaults.
I’m going to look at the ‘Notebook Storage in S3’ steps next and set up credentials – maybe I need ~/.aws/credentials on the master node.
If you know anyone who’s set up a notebook environment in some multitenant fashion to interact with EMR, please let me know!
Thanks for your help
-Ken
From: Jonathan Kelly <jo...@gmail.com>
Reply-To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Date: Friday, May 26, 2017 at 6:56 PM
To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Cc: "users@zeppelin.incubator.apache.org" <us...@zeppelin.incubator.apache.org>
Subject: Re: Zeppelin 0.7.1,config file not being read
I have not tried using Shiro with Zeppelin yet, but one thing I'll point out is that for Zeppelin on EMR, configuration should be done through zeppelin-env.sh rather than zeppelin-site.xml. All Zeppelin configuration properties can be set via either file, so we (I'm on the EMR team, btw) decided to support just one of them. At least, zeppelin-env.sh is the only one you can change via the Configuration JSON you may supply when creating a cluster. Nothing stops you from ssh'ing to the master node and changing the files yourself.
~ Jonathan
On Fri, May 26, 2017 at 6:15 PM Jaideep Singh <ja...@gmail.com>> wrote:
Hello,
I am also new to this but you can try changing
> /api/version = authc,roles[admin]
Thanks,
Jaideep Singh
On Saturday, May 27, 2017, kebarcla <kb...@ancestry.com>> wrote:
Hello,
I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.
In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.
After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?
Thanks for any pointers,
Ken
# shiro.ini changes:
< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Re: Zeppelin 0.7.1,config file not being read
Posted by Jonathan Kelly <jo...@gmail.com>.
I have not tried using Shiro with Zeppelin yet, but one thing I'll point
out is that for Zeppelin on EMR, configuration should be done through
zeppelin-env.sh rather than zeppelin-site.xml. All Zeppelin configuration
properties can be set via either file, so we (I'm on the EMR team, btw)
decided to support just one of them. At least, zeppelin-env.sh is the only
one you can change via the Configuration JSON you may supply when creating
a cluster. Nothing stops you from ssh'ing to the master node and changing
the files yourself.
~ Jonathan
On Fri, May 26, 2017 at 6:15 PM Jaideep Singh <ja...@gmail.com> wrote:
> Hello,
>
> I am also new to this but you can try changing
> > /api/version = authc,roles[admin]
>
> Thanks,
> Jaideep Singh
>
>
>
>
> On Saturday, May 27, 2017, kebarcla <kb...@ancestry.com> wrote:
>
>> Hello,
>>
>> I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
>> data tools deployed. I'd like to set up users to access the cluster.
>>
>> In following the steps documented in 'Apache Shiro authentication for
>> Apache
>> Zeppelin' I'm finding that my changes to conf/shiro.ini or
>> zeppelin-site.xml
>> on the master node seem to be ignored: I don't see them under
>> 'Configuration' on the user drop-down.
>>
>> After restarting Zeppelin server, I'm still automatically logged in as
>> 'anonymous.'
>> How does one log out, and log back in as another user?
>> How to get Zeppelin server to recognize config file changes?
>>
>> Thanks for any pointers,
>> Ken
>>
>> # shiro.ini changes:
>>
>> < newuser = Password!, admin
>> < #/api/version = anon
>> < /api/interpreter/** = authc, roles[admin]
>> < /api/configurations/** = authc, roles[admin]
>> < /api/credential/** = authc, roles[admin]
>> ---
>> > /api/version = anon
>> > #/api/interpreter/** = authc, roles[admin]
>> > #/api/configurations/** = authc, roles[admin]
>> > #/api/credential/** = authc, roles[admin]
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
>> Sent from the Apache Zeppelin Users (incubating) mailing list mailing
>> list archive at Nabble.com.
>>
>
Re: Zeppelin 0.7.1,config file not being read
Posted by Ken Barclay <kb...@ancestry.com>.
Seems logical – I gave it a shot, but it didn’t affect the outcome. I seem to have some fundamental disconnect between my configuration & my running instance. I’m even passing the config directory as an argument:
/usr/lib/zeppelin/bin/zeppelin-daemon.sh --config /etc/zeppelin/conf start
Thanks though Jaideep
From: Jaideep Singh <ja...@gmail.com>
Reply-To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Date: Friday, May 26, 2017 at 6:15 PM
To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Cc: "users@zeppelin.incubator.apache.org" <us...@zeppelin.incubator.apache.org>
Subject: Re: Zeppelin 0.7.1,config file not being read
Hello,
I am also new to this but you can try changing
> /api/version = authc,roles[admin]
Thanks,
Jaideep Singh
On Saturday, May 27, 2017, kebarcla <kb...@ancestry.com>> wrote:
Hello,
I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.
In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.
After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?
Thanks for any pointers,
Ken
# shiro.ini changes:
< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Re: Zeppelin 0.7.1,config file not being read
Posted by Jaideep Singh <ja...@gmail.com>.
Hello,
I am also new to this but you can try changing
> /api/version = authc,roles[admin]
Thanks,
Jaideep Singh
On Saturday, May 27, 2017, kebarcla <kb...@ancestry.com> wrote:
> Hello,
>
> I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
> data tools deployed. I'd like to set up users to access the cluster.
>
> In following the steps documented in 'Apache Shiro authentication for
> Apache
> Zeppelin' I'm finding that my changes to conf/shiro.ini or
> zeppelin-site.xml
> on the master node seem to be ignored: I don't see them under
> 'Configuration' on the user drop-down.
>
> After restarting Zeppelin server, I'm still automatically logged in as
> 'anonymous.'
> How does one log out, and log back in as another user?
> How to get Zeppelin server to recognize config file changes?
>
> Thanks for any pointers,
> Ken
>
> # shiro.ini changes:
>
> < newuser = Password!, admin
> < #/api/version = anon
> < /api/interpreter/** = authc, roles[admin]
> < /api/configurations/** = authc, roles[admin]
> < /api/credential/** = authc, roles[admin]
> ---
> > /api/version = anon
> > #/api/interpreter/** = authc, roles[admin]
> > #/api/configurations/** = authc, roles[admin]
> > #/api/credential/** = authc, roles[admin]
>
>
>
> --
> View this message in context: http://apache-zeppelin-users-
> incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-
> config-file-not-being-read-tp5695.html
> Sent from the Apache Zeppelin Users (incubating) mailing list mailing list
> archive at Nabble.com.
>