You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by bc...@gmail.com on 2009/06/02 00:14:57 UTC
Hive JDBC
I am using a Hive instance using MySQL as the meta-data store. The cluster
works fine in command line mode.
The problem I am having is that when I attempt to connect using the Hive
JDBC connector I get the following error:
java.sql.SQLException:
org.jpox.jdo.exceptions.TransactionNotReadableException: Cant read fields
outside of transactions. You may want to set 'NontransactionalRead=true'.
[java] FailedObject:1[OID]org.apache.hadoop.hive.metastore.model.MDatabase
[java] at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:71)
[java] at java.sql.DriverManager.getConnection(DriverManager.java:582)
[java] at java.sql.DriverManager.getConnection(DriverManager.java:185)
[java] at com.adknowledge.HiveDriver.HiveDrvr.<init>(HiveDrvr.java:27)
[java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:46)
[java] java.lang.NullPointerException
[java] at com.adknowledge.HiveDriver.HiveDrvr.go(HiveDrvr.java:36)
[java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:47)
This is after the basic connection to the meta-store has worked.
.
.
.
. [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 foreign
key(s) for table `PARTITION_KEY_VALS`
[java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 unique key(s)
for table `PARTITION_KEY_VALS`
[java] 09/06/01 17:08:41 INFO Datastore.Schema: Catalog "hive", Schema ""
initialised - managing 14 classes
[java] 09/06/01 17:08:41 INFO metastore.ObjectStore: Initialized ObjectStore
.
.Has anyone tried this combination before or am I in uncharted waters?
Re: Hive JDBC
Posted by Bill Graham <bi...@gmail.com>.
I've been able to get the Hive JDBC client working with a Hive server (I'd
reply to the referenced thread, but I joined the list since then so I don't
have it). I haven't tried using MySQL yet for the metastore, but this is
what my setup looks like. Give this a shot, then you should be able to swap
out MySQL for Derby on the Hive server with a setup similar to this one:
Hadoop setup: 4 node clister running version 0.18.3
Hive install: hive-0.3.0-hadoop-0.18.0-bin
- Hive Server:
I'm running the Hive server on the same host as my namenode, using the
default hive configs. (I have HADOOP_HOME set, but not HADOOP_VERSION or
ANT_HOME.) From my Hive directory I use this command starts the server
listening on the default port 10000:
> bin/hive --service hiveserver
Since I'm using the default configs, this is using the servers local derby
metastore. Initially I was running the JDBC client and the server on the
same host and was running into all sorts of issues. I suspect this was due
to conflicts with the JDBC client and the server accessing some of the same
local Hive resources??
- JDBC client
I've got a groovy test script that tests the basic functionality (add/drop
table, load, select) running on a separate host. Here's a snippet of the
connection initialization:
String uri = "jdbc:hive://myhostname.com:10000/default"; // for standalone
mode
String driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";
Class.forName(driverName);
Connection con = DriverManager.getConnection(uri, "", "");
This worked for me with the default hadoop JDBC jar, but there was a bug
where the first item in the results set was returning in an infinite loop.
This has been fixed on the hive trunk. After checking out the trunk and
building, the new jar fixed this issue. I run the groovy script with the
following bash script, using the patched jar:
export HIVE_HOME=/path/to/hive
export
CLASSPATH=lib-ext/hive_jdbc_snapshot_778799.jar:$HIVE_HOME/lib/hive_exec.jar:$HIVE_HOME/lib/hive_service.jar:$HIVE_HOME/lib/hive_metastore.jar:$HIVE_HOME/lib/libfb303.jar:$HIVE_HOME/lib/log4j-1.2.15.jar:lib-ext/hadoop-0.18.3-core.jar
groovy test_hive_jdbc.groovy
If you want to swap in MySQL for derby for the matastore, you can change the
configs on the server and leave the client as is.
HTH.
Bill
On Mon, Jun 1, 2009 at 4:31 PM, Jeff Hammerbacher <ha...@cloudera.com>wrote:
> Hey,
>
> The current state of the JDBC driver for Hive is probably best described as
> "busted". See the following thread for more:
> http://markmail.org/thread/w6if4fqxgdzxq4r5.
>
> If you have significant expertise with JDBC, drop me a line off list.
>
> Regards,
> Jeff
>
>
> On Mon, Jun 1, 2009 at 3:14 PM, <bc...@gmail.com> wrote:
>
>> I am using a Hive instance using MySQL as the meta-data store. The cluster
>> works fine in command line mode.
>>
>> The problem I am having is that when I attempt to connect using the Hive
>> JDBC connector I get the following error:
>>
>> java.sql.SQLException:
>> org.jpox.jdo.exceptions.TransactionNotReadableException: Cant read fields
>> outside of transactions. You may want to set 'NontransactionalRead=true'.
>> [java] FailedObject:1[OID]org.apache.hadoop.hive.metastore.model.MDatabase
>> [java] at
>> org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:71)
>> [java] at java.sql.DriverManager.getConnection(DriverManager.java:582)
>> [java] at java.sql.DriverManager.getConnection(DriverManager.java:185)
>> [java] at com.adknowledge.HiveDriver.HiveDrvr.<init>(HiveDrvr.java:27)
>> [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:46)
>> [java] java.lang.NullPointerException
>> [java] at com.adknowledge.HiveDriver.HiveDrvr.go(HiveDrvr.java:36)
>> [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:47)
>>
>> This is after the basic connection to the meta-store has worked.
>> .
>> .
>> .
>> . [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 foreign
>> key(s) for table `PARTITION_KEY_VALS`
>> [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 unique key(s)
>> for table `PARTITION_KEY_VALS`
>> [java] 09/06/01 17:08:41 INFO Datastore.Schema: Catalog "hive", Schema ""
>> initialised - managing 14 classes
>> [java] 09/06/01 17:08:41 INFO metastore.ObjectStore: Initialized
>> ObjectStore
>>
>> .
>> .Has anyone tried this combination before or am I in uncharted waters?
>
>
>
Re: Hive JDBC
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
Hey,
The current state of the JDBC driver for Hive is probably best described as
"busted". See the following thread for more:
http://markmail.org/thread/w6if4fqxgdzxq4r5.
If you have significant expertise with JDBC, drop me a line off list.
Regards,
Jeff
On Mon, Jun 1, 2009 at 3:14 PM, <bc...@gmail.com> wrote:
> I am using a Hive instance using MySQL as the meta-data store. The cluster
> works fine in command line mode.
>
> The problem I am having is that when I attempt to connect using the Hive
> JDBC connector I get the following error:
>
> java.sql.SQLException:
> org.jpox.jdo.exceptions.TransactionNotReadableException: Cant read fields
> outside of transactions. You may want to set 'NontransactionalRead=true'.
> [java] FailedObject:1[OID]org.apache.hadoop.hive.metastore.model.MDatabase
> [java] at
> org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:71)
> [java] at java.sql.DriverManager.getConnection(DriverManager.java:582)
> [java] at java.sql.DriverManager.getConnection(DriverManager.java:185)
> [java] at com.adknowledge.HiveDriver.HiveDrvr.<init>(HiveDrvr.java:27)
> [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:46)
> [java] java.lang.NullPointerException
> [java] at com.adknowledge.HiveDriver.HiveDrvr.go(HiveDrvr.java:36)
> [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:47)
>
> This is after the basic connection to the meta-store has worked.
> .
> .
> .
> . [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 foreign
> key(s) for table `PARTITION_KEY_VALS`
> [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 unique key(s)
> for table `PARTITION_KEY_VALS`
> [java] 09/06/01 17:08:41 INFO Datastore.Schema: Catalog "hive", Schema ""
> initialised - managing 14 classes
> [java] 09/06/01 17:08:41 INFO metastore.ObjectStore: Initialized
> ObjectStore
>
> .
> .Has anyone tried this combination before or am I in uncharted waters?