You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by pr...@apache.org on 2017/05/03 15:47:23 UTC
zeppelin git commit: [Zeppelin 2367] Hive JDBC proxy user option
should be available even without kerberos
Repository: zeppelin
Updated Branches:
refs/heads/master 62aec9fbd -> 08a981403
[Zeppelin 2367] Hive JDBC proxy user option should be available even without kerberos
### What is this PR for?
Hive JDBC proxy user option should be available generically.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* [Zeppelin 2367](https://issues.apache.org/jira/browse/ZEPPELIN-2367)
### How should this be tested?
Enable Shiro authentication and set `zeppelin.jdbc.auth.type` as `SIMPLE` in the interpreter setting, and observe the connection string for the Hive.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need an update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? Yes
Author: Prabhjyot Singh <pr...@gmail.com>
Closes #2229 from prabhjyotsingh/ZEPPELIN-2367 and squashes the following commits:
84b5e55b2 [Prabhjyot Singh] add logger.warn for hive and impersonation
45c90a8e2 [Prabhjyot Singh] improve doc
9fee9d2a9 [Prabhjyot Singh] replace hive with generic method
a348e969a [Prabhjyot Singh] revert "zeppelin.jdbc.auth.kerberos.proxy.enable" behaviour
e2bdbb2ad [Prabhjyot Singh] include e as inner exception
c180f5ce2 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367
1802b453f [Prabhjyot Singh] remove hive string from logger
513987a28 [Prabhjyot Singh] apply genric logic to appendProxyUserToURL
3fa2b1e98 [Prabhjyot Singh] change name to appendProxyUserToURL
a75167415 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367
4c382eefa [Prabhjyot Singh] log user details as well
d51e770b2 [Prabhjyot Singh] add doc in jdbc.md
01b18b9d2 [Prabhjyot Singh] add doc (reverted from commit ee8a6b524c481210486761032cb1f5fd6266bb54)
40489c89d [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367
ee8a6b524 [Prabhjyot Singh] add doc
8999d93ae [Prabhjyot Singh] ZEPPELIN-2367: Hive JDBC proxy user option should be avail even without kerberos
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/08a98140
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/08a98140
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/08a98140
Branch: refs/heads/master
Commit: 08a9814033e216eb2d340c49d171f3b66c19eb61
Parents: 62aec9f
Author: Prabhjyot Singh <pr...@gmail.com>
Authored: Sun Apr 30 00:15:19 2017 +0530
Committer: Prabhjyot Singh <pr...@gmail.com>
Committed: Wed May 3 21:17:16 2017 +0530
----------------------------------------------------------------------
docs/interpreter/jdbc.md | 35 ++++++++---
.../apache/zeppelin/jdbc/JDBCInterpreter.java | 66 +++++++++++---------
.../jdbc/security/JDBCSecurityImpl.java | 5 +-
.../interpreter/InterpreterException.java | 4 ++
4 files changed, 68 insertions(+), 42 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/docs/interpreter/jdbc.md
----------------------------------------------------------------------
diff --git a/docs/interpreter/jdbc.md b/docs/interpreter/jdbc.md
index f66ad6d..f10444b 100644
--- a/docs/interpreter/jdbc.md
+++ b/docs/interpreter/jdbc.md
@@ -171,7 +171,7 @@ There are more JDBC interpreter properties you can specify like below.
</tr>
<tr>
<td>zeppelin.jdbc.auth.kerberos.proxy.enable</td>
- <td>When auth type is Kerberos, enable/disable Kerberos proxy with the login user to get the connection. Default value is true.</td>
+ <td>When auth type is Kerberos, enable/disable Kerberos proxy with the login user to get the connection. Default value is true.</td>
</tr>
<tr>
<td>default.jceks.file</td>
@@ -202,7 +202,7 @@ To bind the interpreters created in the interpreter setting page, click the gear
<img src="../assets/themes/zeppelin/img/docs-img/click_interpreter_binding_button.png" width="600px" />
-Select(blue) or deselect(white) the interpreter buttons depending on your use cases.
+Select(blue) or deselect(white) the interpreter buttons depending on your use cases.
If you need to use more than one interpreter in the notebook, activate several buttons.
Don't forget to click `Save` button, or you will face `Interpreter *** is not found` error.
@@ -285,7 +285,7 @@ An example settings of interpreter for the two data sources, each of which has i
</table>
##### Usage
-Test of execution *precode* for each data source.
+Test of execution *precode* for each data source.
```sql
%jdbc
@@ -480,7 +480,7 @@ Here are some examples you can refer to. Including the below connectors, you can
[Maven Repository: com.amazonaws:aws-java-sdk-redshift](https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-redshift)
-### Apache Hive
+### Apache Hive
<img src="../assets/themes/zeppelin/img/docs-img/hive_setting.png" width="600px" />
@@ -507,12 +507,11 @@ Here are some examples you can refer to. Including the below connectors, you can
<td>hive_password</td>
</tr>
<tr>
- <td>hive.proxy.user</td>
- <td>true or false</td>
+ <td>default.proxy.user.property</td>
+ <td>Example value: hive.server2.proxy.user</td>
+ </tr>
</table>
-Connection to Hive JDBC with a proxy user can be disabled with `hive.proxy.user` property (set to true by default)
-
[Apache Hive 1 JDBC Driver Docs](https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC)
[Apache Hive 2 JDBC Driver Docs](https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC)
@@ -534,6 +533,26 @@ Connection to Hive JDBC with a proxy user can be disabled with `hive.proxy.user`
[Maven Repository : org.apache.hive:hive-jdbc](https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc)
+##### Impersonation
+When Zeppelin server is running with authentication enabled, then the interpreter can utilize Hive's user proxy feature i.e. send extra parameter for creating and running a session ("hive.server2.proxy.user=": "${loggedInUser}"). This is particularly useful when multiple users are sharing a notebooks.
+
+To enable this set following:
+ - `zeppelin.jdbc.auth.type` as `SIMPLE` or `KERBEROS` (if required) in the interpreter setting.
+ - `${prefix}.proxy.user.property` as `hive.server2.proxy.user`
+ Example configuration
+
+ *Properties*
+
+ | name | value |
+ |:------------------------- |:--------------------------------------------------------------------------------------------------|
+ | hive.driver | org.apache.hive.jdbc.HiveDriver |
+ | hive.password | |
+ | hive.url | jdbc:hive2://hive-server-host:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 |
+ | hive.proxy.user.property | hive.proxy.user.property |
+ | zeppelin.jdbc.auth.type | SIMPLE |
+
+
+
### Apache Phoenix
Phoenix supports `thick` and `thin` connection types:
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
----------------------------------------------------------------------
diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
index 95b2a1a..3272bc1 100644
--- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
+++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
@@ -26,7 +26,6 @@ import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
-import java.util.Map;
import java.util.Properties;
import java.util.Set;
@@ -179,10 +178,6 @@ public class JDBCInterpreter extends Interpreter {
}
logger.debug("JDBC PropretiesMap: {}", basePropretiesMap);
- if (!isEmpty(property.getProperty("zeppelin.jdbc.auth.type"))) {
- JDBCSecurityImpl.createSecureConfiguration(property);
- }
-
setMaxLineResults();
}
@@ -355,36 +350,25 @@ public class JDBCInterpreter extends Interpreter {
} else {
UserGroupInformation.AuthenticationMethod authType = JDBCSecurityImpl.getAuthtype(property);
+ final String connectionUrl = appendProxyUserToURL(url, user, propertyKey);
+
+ JDBCSecurityImpl.createSecureConfiguration(property, authType);
switch (authType) {
case KERBEROS:
if (user == null || "false".equalsIgnoreCase(
- property.getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable"))) {
- connection = getConnectionFromPool(url, user, propertyKey, properties);
+ property.getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable"))) {
+ connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties);
} else {
- if (url.trim().startsWith("jdbc:hive")) {
- StringBuilder connectionUrl = new StringBuilder(url);
- Integer lastIndexOfUrl = connectionUrl.indexOf("?");
- if (lastIndexOfUrl == -1) {
- lastIndexOfUrl = connectionUrl.length();
- }
- boolean hasProxyUser = property.containsKey("hive.proxy.user");
- if (!hasProxyUser || !property.getProperty("hive.proxy.user").equals("false")){
- logger.debug("Using hive proxy user");
- connectionUrl.insert(lastIndexOfUrl, ";hive.server2.proxy.user=" + user + ";");
- }
- connection = getConnectionFromPool(connectionUrl.toString(),
- user, propertyKey, properties);
+ if (basePropretiesMap.get(propertyKey).containsKey("proxy.user.property")) {
+ connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties);
} else {
UserGroupInformation ugi = null;
try {
ugi = UserGroupInformation.createProxyUser(
- user, UserGroupInformation.getCurrentUser());
+ user, UserGroupInformation.getCurrentUser());
} catch (Exception e) {
logger.error("Error in getCurrentUser", e);
- StringBuilder stringBuilder = new StringBuilder();
- stringBuilder.append(e.getMessage()).append("\n");
- stringBuilder.append(e.getCause());
- throw new InterpreterException(stringBuilder.toString());
+ throw new InterpreterException("Error in getCurrentUser", e);
}
final String poolKey = propertyKey;
@@ -392,28 +376,48 @@ public class JDBCInterpreter extends Interpreter {
connection = ugi.doAs(new PrivilegedExceptionAction<Connection>() {
@Override
public Connection run() throws Exception {
- return getConnectionFromPool(url, user, poolKey, properties);
+ return getConnectionFromPool(connectionUrl, user, poolKey, properties);
}
});
} catch (Exception e) {
logger.error("Error in doAs", e);
- StringBuilder stringBuilder = new StringBuilder();
- stringBuilder.append(e.getMessage()).append("\n");
- stringBuilder.append(e.getCause());
- throw new InterpreterException(stringBuilder.toString());
+ throw new InterpreterException("Error in doAs", e);
}
}
}
break;
default:
- connection = getConnectionFromPool(url, user, propertyKey, properties);
+ connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties);
}
}
return connection;
}
+ private String appendProxyUserToURL(String url, String user, String propertyKey) {
+ StringBuilder connectionUrl = new StringBuilder(url);
+
+ if (user != null && !user.equals("anonymous") &&
+ basePropretiesMap.get(propertyKey).containsKey("proxy.user.property")) {
+
+ Integer lastIndexOfUrl = connectionUrl.indexOf("?");
+ if (lastIndexOfUrl == -1) {
+ lastIndexOfUrl = connectionUrl.length();
+ }
+ logger.info("Using proxy user as :" + user);
+ logger.info("Using proxy property for user as :" +
+ basePropretiesMap.get(propertyKey).getProperty("proxy.user.property"));
+ connectionUrl.insert(lastIndexOfUrl, ";" +
+ basePropretiesMap.get(propertyKey).getProperty("proxy.user.property") + "=" + user + ";");
+ } else if (user != null && !user.equals("anonymous") && url.contains("hive")) {
+ logger.warn("User impersonation for hive has changed please refer: http://zeppelin.apache" +
+ ".org/docs/latest/interpreter/jdbc.html#apache-hive");
+ }
+
+ return connectionUrl.toString();
+ }
+
private String getPassword(Properties properties) throws IOException {
if (isNotEmpty(properties.getProperty(PASSWORD_KEY))) {
return properties.getProperty(PASSWORD_KEY);
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
----------------------------------------------------------------------
diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
index 32a7990..25959e1 100644
--- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
+++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
@@ -38,9 +38,8 @@ public class JDBCSecurityImpl {
/***
* @param properties
*/
- public static void createSecureConfiguration(Properties properties) {
- AuthenticationMethod authType = getAuthtype(properties);
-
+ public static void createSecureConfiguration(Properties properties,
+ AuthenticationMethod authType) {
switch (authType) {
case KERBEROS:
Configuration conf = new
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
----------------------------------------------------------------------
diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
index 30c1c0a..ebd184e 100644
--- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
+++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
@@ -31,4 +31,8 @@ public class InterpreterException extends RuntimeException {
super(m);
}
+ public InterpreterException(String msg, Throwable t) {
+ super(msg, t);
+ }
+
}