You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by prabhjyotsingh <gi...@git.apache.org> on 2016/07/09 13:20:07 UTC
[GitHub] zeppelin pull request #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter shoul...
GitHub user prabhjyotsingh opened a pull request:
https://github.com/apache/zeppelin/pull/1157
[ZEPPELIN-1146] Zeppelin JDBC interpreter should work in a Kerberos environment
### What is this PR for?
Zeppelin JDBC interpreter should work in a Kerberos environment
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Update doc
### What is the Jira issue?
* [ZEPPELIN-1146](https://issues.apache.org/jira/browse/ZEPPELIN-1146)
### How should this be tested?
In JDBC interpreter setting (say hive) add following properties
- hive.auth.type = KERBEROS
- hive.principal = hive's principal
- hive.keytab.location = keytab location
Now try and run any of hive's query (say `show tables`) it should return with valid results.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/prabhjyotsingh/zeppelin ZEPPELIN-1146
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1157.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1157
----
commit c2313e4c64ff015f71c9b837329540fe1ff9e339
Author: Prabhjyot Singh <pr...@gmail.com>
Date: 2016-07-09T13:19:42Z
ZEPPELIN-1146 Zeppelin JDBC interpreter should work in a Kerberos environment
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
You can either use %hive or %jdbc(hive), for the same.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by rconlinehwx <gi...@git.apache.org>.
Github user rconlinehwx commented on the issue:
https://github.com/apache/zeppelin/pull/1157
LGTM.
@jongyoul Typically I use an ambari created 3 node cluster and then follow these instructions to setup my kerberos - https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Security_Guide/content/ch_configuring_amb_hdp_for_kerberos.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
Merging this if no more discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@prabhjyotsingh Changes look good, but do you guide me to this PR with my local machine? Does it have easy way to test it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@jongyoul i couldn't figure out a easier way, than having a actual kerberos environment, and then testing it there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@rja1 Hi, have you solved this error?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@prabhjyotsingh Thanks for checking it. I think we should fix all of flaky tests
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@prabhjyotsingh This PR is not passed. Why do you merge it? I think we need to pass the CI even though we have some flaky tests. Could you please check it again?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@rja1 I think I'm able to reproduce the same error as you are getting, and think it is because of ";" at the end of query that you are executing, remove that ";" and it should work.
With ";" I too get this on running `select * from table_name;`
```
Error while compiling statement: FAILED: ParseException line 1:17 cannot recognize input near 'table_name' ';' '<EOF>' in from source 0
class org.apache.hive.service.cli.HiveSQLException
org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:258)
```
@jongyoul, right now build passes https://travis-ci.org/apache/zeppelin/builds/144647358, but from next time, I'll re-trigger of flaky test failures.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
I think I have seen this error before
```
org.apache.hadoop.hbase.MasterNotRunningException:
com.google.protobuf.ServiceException:
org.apache.hadoop.hbase.exceptions.ConnectionClosingException
```
This is usually because of either phoenix or hbase version; try taking phoenix-\*-4.6.0-* and/or hbase-client-1.1.
I would recommend instead of all of these
```
/opt/cloudera/CDH/jars/phoenix-core-4.7.0-cdh5.5.1.jar
/opt/cloudera/CDH/jars/hbase-common-1.0.0-cdh5.5.2.jar
/opt/cloudera/CDH/jars/hbase-client-1.0.0-cdh5.5.2.jar
/opt/cloudera/CDH/jars/phoenix-4.7.0-cdh5.5.1-client.jar
```
try `org.apache.phoenix:phoenix-core:4.5.0-HBase-1.1` or `org.apache.phoenix:phoenix-core:4.6.0-HBase-1.1` in dependencies, for testing, and once it works you may go back to respective cdh jars.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@jongyoul Sure let me check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
Yes fair point, let me do a quick test and do it right away.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by rja1 <gi...@git.apache.org>.
Github user rja1 commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@prabhjyotsingh would you mind sharing your jdbc interpreter settings please? I must be missing something.. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by rja1 <gi...@git.apache.org>.
Github user rja1 commented on the issue:
https://github.com/apache/zeppelin/pull/1157
Thanks @prabhjyotsingh. Novice question: how to you make the distinction in the notebook that you want jdbc hive vs jdbc postgresql, jdbc phoenix, etc..
If I run:
%jdbc
show tables;
Zeppelin defaults to trying to connect to postgres (as that's the default driver, which makes sense).
I'm guessing you're doing something like:
%jdbc.hive?
show tables;
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@rja1 Sure, attaching both screenshot and json. I have tested this for hive and phoenix.
<img width="1428" alt="screen shot 2016-07-12 at 11 03 57 pm" src="https://cloud.githubusercontent.com/assets/674497/16776893/11020f28-4885-11e6-878b-d715561efca3.png">
```
{
"id": "2BQPN9U57",
"name": "jdbc",
"group": "jdbc",
"properties": {
"phoenix.user": "phoenixuser",
"hive.url": "jdbc:hive2://prabhu-zeppelin-secure-1.novalocal:2181,prabhu-zeppelin-secure-2.novalocal:2181,prabhu-zeppelin-secure-4.novalocal:2181/;serviceDiscoveryMode\u003dzooKeeper;zooKeeperNamespace\u003dhiveserver2",
"psql.password": "",
"default.driver": "org.postgresql.Driver",
"phoenix.driver": "org.apache.phoenix.jdbc.PhoenixDriver",
"hive.user": "hive",
"psql.user": "phoenixuser",
"psql.url": "jdbc:postgresql://localhost:5432/",
"default.user": "gpadmin",
"phoenix.hbase.client.retries.number": "1",
"phoenix.url": "jdbc:phoenix:prabhu-zeppelin-secure-1.novalocal,prabhu-zeppelin-secure-2.novalocal,prabhu-zeppelin-secure-4.novalocal:/hbase-secure",
"tajo.url": "jdbc:tajo://localhost:26002/default",
"tajo.driver": "org.apache.tajo.jdbc.TajoDriver",
"psql.driver": "org.postgresql.Driver",
"default.password": "",
"zeppelin.interpreter.localRepo": "/usr/hdp/current/zeppelin-server/local-repo/2BQPN9U57",
"zeppelin.jdbc.auth.type": "KERBEROS",
"hive.password": "",
"zeppelin.jdbc.concurrent.use": "true",
"hive.driver": "org.apache.hive.jdbc.HiveDriver",
"zeppelin.jdbc.keytab.location": "/etc/security/keytabs/zeppelin.server.kerberos.keytab",
"common.max_count": "1000",
"phoenix.password": "",
"zeppelin.jdbc.principal": "zeppelin/securrr@EXAMPLE.COM",
"zeppelin.jdbc.concurrent.max_connection": "10",
"default.url": "jdbc:postgresql://localhost:5432/"
},
"interpreterGroup": [{
"class": "org.apache.zeppelin.jdbc.JDBCInterpreter",
"name": "sql"
}],
"dependencies": [{
"groupArtifactVersion": "org.apache.hive:hive-jdbc:2.0.1",
"local": false
}, {
"groupArtifactVersion": "org.apache.hadoop:hadoop-common:2.7.2",
"local": false
}, {
"groupArtifactVersion": "org.apache.hive.shims:hive-shims-0.23:2.1.0",
"local": false
}, {
"groupArtifactVersion": "org.apache.phoenix:phoenix-core:4.4.0-HBase-1.1",
"local": false
}],
"option": {
"remote": true,
"perNoteSession": false,
"perNoteProcess": false,
"isExistingProcess": false
}
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by rconline <gi...@git.apache.org>.
Github user rconline commented on the issue:
https://github.com/apache/zeppelin/pull/1157
@jongyoul i think this suggestion makes sense. Its possible that other jdbc data sources may have security setup via ACL and don't need Kerberos.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by rja1 <gi...@git.apache.org>.
Github user rja1 commented on the issue:
https://github.com/apache/zeppelin/pull/1157
Thanks for the tip @prabhjyotsingh and for your work, hive is working!
I'm running into an issue with jdbc(phoenix) now though, hoping you can help. My interpreter config is listed below. Note that we don't have a /hbase-secure dir in zookeeper, just hbase.
[zk: localhost:2181(CONNECTED) 1] ls /hbase
[replication, meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, region-in-transition, online-snapshot, acl, master, running, balancer, tokenauth, recovering-regions, draining, namespace, hbaseid, table]
Here's the notebook / error:
select * from USER_ACCOUNTS where USER_SEED = '1000'
Failed after attempts=1, exceptions:
Mon Jul 18 14:00:22 MDT 2016, RpcRetryingCaller{globalStartTime=1468872022128, pause=100, retries=1}, org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to name01.hadoop.test.company.com/10.4.59.25:60000 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to name01.hadoop.test.company.com/10.4.59.25:60000 is closing. Call id=0, waitTime=11
class org.apache.phoenix.exception.PhoenixIOException
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:111)
org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1064)
org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1370)
org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2116)
org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828)
org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338)
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326)
org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324)
org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1326)
org.apache.phoenix.query.ConnectionQueryServicesImpl$13.call(ConnectionQueryServicesImpl.java:2275)
org.apache.phoenix.query.ConnectionQueryServicesImpl$13.call(ConnectionQueryServicesImpl.java:2244)
org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:78)
org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2244)
org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:233)
org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:135)
org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:202)
java.sql.DriverManager.getConnection(DriverManager.java:664)
java.sql.DriverManager.getConnection(DriverManager.java:208)
org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:226)
org.apache.zeppelin.jdbc.JDBCInterpreter.getStatement(JDBCInterpreter.java:237)
org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:296)
org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:402)
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:383)
org.apache.zeppelin.scheduler.Job.run(Job.java:176)
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Interpreter Config:
"2BRGRRCBW": {
"id": "2BRGRRCBW",
"name": "jdbc",
"group": "jdbc",
"properties": {
"phoenix.user": "zeppelin",
"hive.url": "jdbc:hive2://cms01.hadoop.test.company.com:10000/default;principal\u003dhive/_HOST@HADOOP.TEST.company.COM",
"default.driver": "org.postgresql.Driver",
"phoenix.driver": "org.apache.phoenix.jdbc.PhoenixDriver",
"hive.user": "hive",
"psql.password": "",
"psql.user": "phoenixuser",
"psql.url": "jdbc:postgresql://localhost:5432/",
"default.user": "gpadmin",
"phoenix.hbase.client.retries.number": "1",
"phoenix.url": "jdbc:phoenix:cms01.hadoop.test.company.com:/hbase",
"tajo.url": "jdbc:tajo://localhost:26002/default",
"tajo.driver": "org.apache.tajo.jdbc.TajoDriver",
"psql.driver": "org.postgresql.Driver",
"default.password": "",
"zeppelin.interpreter.localRepo": "/opt/zeppelin-ZEPPELIN-1146/local-repo/2BRGRRCBW",
"zeppelin.jdbc.auth.type": "KERBEROS",
"zeppelin.jdbc.concurrent.use": "true",
"hive.password": "",
"hive.driver": "org.apache.hive.jdbc.HiveDriver",
"zeppelin.jdbc.keytab.location": "/opt/zeppelin-ZEPPELIN-1146/conf/zeppelin.keytab",
"common.max_count": "1000",
"phoenix.password": "",
"zeppelin.jdbc.principal": "zeppelin@HADOOP.TEST.company.COM",
"zeppelin.jdbc.concurrent.max_connection": "10",
"default.url": "jdbc:postgresql://localhost:5432/"
},
"interpreterGroup": [
{
"name": "sql",
"class": "org.apache.zeppelin.jdbc.JDBCInterpreter"
}
],
"dependencies": [
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hive-jdbc-1.1.0-cdh5.5.2.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hadoop-common-2.6.0-cdh5.5.2.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hive-shims-0.23-1.1.0-cdh5.5.2.jar",
"local": false,
"exclusions": []
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hive-jdbc-1.1.0-cdh5.5.2-standalone.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/phoenix-core-4.7.0-cdh5.5.1.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hbase-common-1.0.0-cdh5.5.2.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hbase-client-1.0.0-cdh5.5.2.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/phoenix-4.7.0-cdh5.5.1-client.jar",
"local": false
}
],
"option": {
"remote": true,
"port": -1,
"perNoteSession": false,
"perNoteProcess": false,
"isExistingProcess": false
}
},
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by rja1 <gi...@git.apache.org>.
Github user rja1 commented on the issue:
https://github.com/apache/zeppelin/pull/1157
Thanks for your work @prabhjyotsingh... Still unable to get this to work with hive (haven't tried phoenix yet)... Looks like it's related to the hive.url, doesn't appear that we have discovery mode enabled. Any direction you can give is much appreciated..
hive.url: jdbc:hive2://data03.hadoop.test.company.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper
hive.url: jdbc:hive2://cms02.hadoop.test.company.com:10000
Error: Could not open client transport with JDBC Uri: jdbc:hive2://cms02.hadoop.test.company.com:10000: Peer indicated failure: Unsupported mechanism type PLAIN
hive.url: jdbc:hive2://cms02.hadoop.test.company.com:10000/default;principal=hive/_HOST@HADOOP.TEST.COMPANY.COM
Error while compiling statement: FAILED: ParseException line 1:11 extraneous input ';' expecting EOF near '<EOF>'
Settings:
"2BRGRRCBW": {
"id": "2BRGRRCBW",
"name": "jdbc",
"group": "jdbc",
"properties": {
"phoenix.user": "phoenixuser",
"hive.url": "jdbc:hive2://cms02.hadoop.test.company.com:10000/default;principal=hive/_HOST@HADOOP.TEST.COMPANY.COM",
"default.driver": "org.postgresql.Driver",
"phoenix.driver": "org.apache.phoenix.jdbc.PhoenixDriver",
"hive.user": "hive",
"psql.password": "",
"psql.user": "phoenixuser",
"psql.url": "jdbc:postgresql://localhost:5432/",
"default.user": "gpadmin",
"phoenix.hbase.client.retries.number": "1",
"phoenix.url": "jdbc:phoenix:localhost:2181:/hbase-unsecure",
"tajo.url": "jdbc:tajo://localhost:26002/default",
"tajo.driver": "org.apache.tajo.jdbc.TajoDriver",
"psql.driver": "org.postgresql.Driver",
"default.password": "",
"zeppelin.interpreter.localRepo": "/opt/zeppelin-ZEPPELIN-1146/local-repo/2BRGRRCBW",
"zeppelin.jdbc.auth.type": "KERBEROS",
"hive.password": "",
"zeppelin.jdbc.concurrent.use": "true",
"hive.driver": "org.apache.hive.jdbc.HiveDriver",
"common.max_count": "1000",
"zeppelin.jdbc.keytab.location": "/opt/zeppelin-ZEPPELIN-1146/conf/zeppelin.keytab",
"phoenix.password": "",
"zeppelin.jdbc.principal": "zeppelin@HADOOP.TEST.COMPANY.COM",
"zeppelin.jdbc.concurrent.max_connection": "10",
"default.url": "jdbc:postgresql://localhost:5432/"
},
"interpreterGroup": [
{
"name": "sql",
"class": "org.apache.zeppelin.jdbc.JDBCInterpreter"
}
],
"dependencies": [
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hive-jdbc-1.1.0-cdh5.5.2.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hadoop-common-2.6.0-cdh5.5.2.jar",
"local": false
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hive-shims-0.23-1.1.0-cdh5.5.2.jar",
"local": false,
"exclusions": []
},
{
"groupArtifactVersion": "/opt/cloudera/CDH/jars/hive-jdbc-1.1.0-cdh5.5.2-standalone.jar",
"local": false
}
],
"option": {
"remote": true,
"port": -1,
"perNoteSession": false,
"perNoteProcess": false,
"isExistingProcess": false
}
I'm then testing the notebook as follows:
%jdbc(hive)
show tables;
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter shoul...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1157
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1157
Doesn't it become potential problem to set hadoop version as 2.7.2? Is there a way to fit the version of hadoop globally?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1157: [ZEPPELIN-1146] Zeppelin JDBC interpreter should work ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1157
And It's not necessary to have hadoop dependencies in case not to depend on hadoop environment like mysql and postresql. I think we need different way to deal with it. I think it's better to change the scope of those dependencies as provided and Zeppelin handles it with reflection. In case that you use Hive, you already have a dependencies of hadoop-common thus it's OK, otherwise, JdbcInterpreter doesn't need to have hadoop-common. What do you think of my idea?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---