You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by rconline <gi...@git.apache.org> on 2016/07/11 14:41:41 UTC
[GitHub] zeppelin pull request #1163: [ZEPPELIN-1149] %sh interpreter kerberos suppor...
GitHub user rconline opened a pull request:
https://github.com/apache/zeppelin/pull/1163
[ZEPPELIN-1149] %sh interpreter kerberos support
### What is this PR for?
Zeppelin Shell interpreter should work in a Kerberos environment
### What type of PR is it?
[Feature]
### What is the Jira issue?
* [ZEPPELIN-1149](https://issues.apache.org/jira/browse/ZEPPELIN-1149)
### How should this be tested?
In JDBC interpreter setting add following properties
- shell.auth.type = KERBEROS
- shell.principal = principal value
- shell.keytab.location = keytab location
Now try and run any shell command (example: hdfs dfs -ls) it should return with valid results.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rconline/zeppelin ZEPPELIN-1149
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1163.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1163
----
commit 5190791c8fedb9978f1e740d026ceef58ae93146
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-11T14:35:40Z
[ZEPPELIN-1149] %sh interpreter kerberos support
commit bc71051f96c7858e09202edef746513e05cf6d77
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-11T14:39:34Z
shell interpreter doc
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1163
Sorry, I mistake it as hdfs interpreter. For shell interpreter, do we must have kerberos support ? I think here kinit is just for the hadoop command, so actually it is for running hdfs command in secured cluster rather than running general shell command in secured cluster.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1163
agree with @felixcheung, use the zeppelin service keytab/principal might bring some other potential security problems. I think user A would expect to run shell script as himself, but actually it run as user zeppelin. And seems all the users are sharing the same principal if I understand correctly.
Besides that I have one question about shiro of zeppelin, does shiro will map the username to hadoop side ? Say I login zeppelin as user A, would that guaranteed there should also be a user A in the hadoop ecosystem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1163: [ZEPPELIN-1149] %sh interpreter kerberos suppor...
Posted by rconline <gi...@git.apache.org>.
GitHub user rconline reopened a pull request:
https://github.com/apache/zeppelin/pull/1163
[ZEPPELIN-1149] %sh interpreter kerberos support
### What is this PR for?
Zeppelin Shell interpreter should work in a Kerberos environment
### What type of PR is it?
[Feature]
### What is the Jira issue?
* [ZEPPELIN-1149](https://issues.apache.org/jira/browse/ZEPPELIN-1149)
### How should this be tested?
In JDBC interpreter setting add following properties
- shell.auth.type = KERBEROS
- shell.principal = principal value
- shell.keytab.location = keytab location
Now try and run any shell command (example: hdfs dfs -ls) it should return with valid results.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rconline/zeppelin ZEPPELIN-1149
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1163.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1163
----
commit 5190791c8fedb9978f1e740d026ceef58ae93146
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-11T14:35:40Z
[ZEPPELIN-1149] %sh interpreter kerberos support
commit 0a77e801e7166c89f5e6ef9d70251cc4f547e5ef
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-11T14:39:34Z
shell interpreter doc
commit b0300e89e3ab5b85ec60cc817d0ff6ea895b7269
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T06:03:58Z
changing property name to "zeppelin.*" to make it match with rest of the project.
commit 60e6d21b5b7b4ac2bfbfe0e7353dbe6130db7fe9
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T06:36:49Z
Addressing feedback from felixcheung
commit 9143b47751e4dfe73495395fd922f219e6bcc665
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T10:18:47Z
Removed Hadoop common dependency and changed switch to String based Kerberos/Simple.
commit 1924b99d3ffc273d8690faed1e2b199d7c1c9cd3
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T17:57:38Z
Merge remote-tracking branch 'origin/master' into ZEPPELIN-1149
# Conflicts:
# docs/interpreter/shell.md
commit 3648a1c19da8343c11557075dbb654a71f781948
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T18:00:59Z
fix CI, resolve merge conflict
commit f6af0f6826bdfc5179d40a3abb35d6fc462534f7
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-14T06:19:43Z
typo fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1163: [ZEPPELIN-1149] %sh interpreter kerberos suppor...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1163
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1163: [ZEPPELIN-1149] %sh interpreter kerberos suppor...
Posted by rconline <gi...@git.apache.org>.
Github user rconline closed the pull request at:
https://github.com/apache/zeppelin/pull/1163
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by rconline <gi...@git.apache.org>.
Github user rconline commented on the issue:
https://github.com/apache/zeppelin/pull/1163
Re-trigger CI
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by rconline <gi...@git.apache.org>.
Github user rconline commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@felixcheung as of now the interpreters are reusing zeppelin keytab. Typically this is case in the hadoop security world, where the keytab is generated at a process level, whereas the authorization is done through impersonation. This is then mapped to the user permissions setup on the underlying data system via posix users. Mature systems which implement full support for security + impersonation, can provide this ability. Right now with this change we get to the first stage which is security. To get to impersonation we will have to start the shell with the user's identity. I think that can be followup patch, in the next couple of weeks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by rconline <gi...@git.apache.org>.
Github user rconline commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@jongyoul no there isn't. Here's the snippet from UserGroupInformation, which is essentially from Hadoop security. In future we will implement multiple methods of security. At present for Kerberos security Kinit is a necessary step. I intend to generalize this across other interpreters as well subsequently.
`public static enum AuthenticationMethod {
SIMPLE,
KERBEROS,
TOKEN,
CERTIFICATE,
KERBEROS_SSL,
PROXY;
`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1163
If each interpreter has a separate settings for Kerberos, eg. shell.keytab.location, would it be harder to manage?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@rconline In case of not having `kinit`, there isn't a way to use kerberos, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1163: [ZEPPELIN-1149] %sh interpreter kerberos suppor...
Posted by rconline <gi...@git.apache.org>.
GitHub user rconline reopened a pull request:
https://github.com/apache/zeppelin/pull/1163
[ZEPPELIN-1149] %sh interpreter kerberos support
### What is this PR for?
Zeppelin Shell interpreter should work in a Kerberos environment
### What type of PR is it?
[Feature]
### What is the Jira issue?
* [ZEPPELIN-1149](https://issues.apache.org/jira/browse/ZEPPELIN-1149)
### How should this be tested?
In JDBC interpreter setting add following properties
- shell.auth.type = KERBEROS
- shell.principal = principal value
- shell.keytab.location = keytab location
Now try and run any shell command (example: hdfs dfs -ls) it should return with valid results.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rconline/zeppelin ZEPPELIN-1149
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1163.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1163
----
commit 5190791c8fedb9978f1e740d026ceef58ae93146
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-11T14:35:40Z
[ZEPPELIN-1149] %sh interpreter kerberos support
commit 0a77e801e7166c89f5e6ef9d70251cc4f547e5ef
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-11T14:39:34Z
shell interpreter doc
commit b0300e89e3ab5b85ec60cc817d0ff6ea895b7269
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T06:03:58Z
changing property name to "zeppelin.*" to make it match with rest of the project.
commit 60e6d21b5b7b4ac2bfbfe0e7353dbe6130db7fe9
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T06:36:49Z
Addressing feedback from felixcheung
commit 9143b47751e4dfe73495395fd922f219e6bcc665
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T10:18:47Z
Removed Hadoop common dependency and changed switch to String based Kerberos/Simple.
commit 1924b99d3ffc273d8690faed1e2b199d7c1c9cd3
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T17:57:38Z
Merge remote-tracking branch 'origin/master' into ZEPPELIN-1149
# Conflicts:
# docs/interpreter/shell.md
commit 3648a1c19da8343c11557075dbb654a71f781948
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-12T18:00:59Z
fix CI, resolve merge conflict
commit f6af0f6826bdfc5179d40a3abb35d6fc462534f7
Author: Rohit Choudhary <rc...@gmail.com>
Date: 2016-07-14T06:19:43Z
typo fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1163
CI fail is unrelated, which fails for Build #6048.2 with `No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.`
Will merge this if no more discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1163: [ZEPPELIN-1149] %sh interpreter kerberos suppor...
Posted by rconline <gi...@git.apache.org>.
Github user rconline closed the pull request at:
https://github.com/apache/zeppelin/pull/1163
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1163
LGTM, tested on ubuntu 14.04 server.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1163
This will also, allow CLI/shell interaction to other data systems such hbase, falcon, hive, kafka, oozie, sqoop, etc. on secured clusters. Hence this is a need, particularly during data ingestion via a notebook.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1163
I think getting impersonation is an important next step that could come after.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@rconline What is `kinit`? I've tried to find a `kinit` in my CentOS, but I couldn't find it. Is it fine?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by rconline <gi...@git.apache.org>.
Github user rconline commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@felixcheung over a period of time, each interpreter will start to have their own Keytab's. I think we should have separate keytabs for different interpreters. We will ofcourse have to invest on getting a better management console for interpreters themselves.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1163
Merging this if no more discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@prabhjyotsingh Thanks for clarification.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1163: [ZEPPELIN-1149] %sh interpreter kerberos support
Posted by rconline <gi...@git.apache.org>.
Github user rconline commented on the issue:
https://github.com/apache/zeppelin/pull/1163
@jongyoul here are some reference - http://linux.die.net/man/1/kinit & http://www.tutorialspoint.com/unix_commands/kinit.htm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---