Issue in using drill JDBC jar in Java code for Hive storage

[Devender Yadav <de...@impetus.co.in>]

Hi,



I am using drill in embedded mode. I added plugin configuration for hive:

{
"type": "hive",
"enabled": true,
"configProps":

{ "hive.metastore.uris": "", "javax.jdo.option.ConnectionURL": "jdbc:mysql://localhost:3306/metastore", "javax.jdo.option.ConnectionDriverName": "com.mysql.jdbc.Driver", "javax.jdo.option.ConnectionUserName": "root", "javax.jdo.option.ConnectionPassword": "root", "hive.metastore.warehouse.dir": "/user/hive/warehouse", "fs.default.name": "file:///", "hive.metastore.sasl.enabled": "false" }

}

Useful portion of my code:

Connection conn = new Driver().connect("jdbc:drill:zk=local",null);
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery("show databases");
while (rs.next())

{ String SCHEMA_NAME = rs.getString("SCHEMA_NAME"); System.out.println(SCHEMA_NAME); }

Exception:

java.sql.SQLException: Failure in starting embedded Drillbit: java.lang.RuntimeException: Unable to deserialize "/tmp/drill/sys.storage_plugins/hive.sys.drill"
at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:109)
at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at org.apache.drill.jdbc.Driver.connect(Driver.java:78)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at com.mkyong.App.main(App.java:28)
Caused by: java.lang.RuntimeException: Unable to deserialize "/tmp/drill/sys.storage_plugins/hive.sys.drill"
at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:140)
at org.apache.drill.exec.store.sys.local.FilePStore$Iter$DeferredEntry.getValue(FilePStore.java:219)
at org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:168)
at org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:132)
at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:244)
at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
... 7 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id 'hive' into a subtype of [simple type, class org.apache.drill.common.logical.StoragePluginConfig]
at [Source: [B@c64ff75; line: 2, column: 3]
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:849)
at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41)
at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1269)
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:912)
at org.apache.drill.exec.store.sys.serialize.JacksonSerializer.deserialize(JacksonSerializer.java:44)
at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:138)
... 12 more


?


Regards,
Devender

________________________________






NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.

Re: Issue in using drill JDBC jar in Java code for Hive storage

[rahul challapalli <ch...@gmail.com>]

I believe this has nothing to do with JDBC in particular. Your hive storage
plugin info seems to be corrupted on your workstation. From the error
message it looks like the drillbit itself failed to start.

Can you backup "/tmp/drill/sys.storage_plugins/hive.sys.drill". Now delete
the "/tmp/drill/sys.storage_plugins/hive.sys.drill" folder. From the UI try
re-creating the hive storage plugin again and see if that changes anything

- Rahul

On Fri, Aug 14, 2015 at 1:26 AM, Devender Yadav <
devender.yadav@impetus.co.in> wrote:

> Hi,
>
>
>
> I am using drill in embedded mode. I added plugin configuration for hive:
>
> {
> "type": "hive",
> "enabled": true,
> "configProps":
>
> { "hive.metastore.uris": "", "javax.jdo.option.ConnectionURL":
> "jdbc:mysql://localhost:3306/metastore",
> "javax.jdo.option.ConnectionDriverName": "com.mysql.jdbc.Driver",
> "javax.jdo.option.ConnectionUserName": "root",
> "javax.jdo.option.ConnectionPassword": "root",
> "hive.metastore.warehouse.dir": "/user/hive/warehouse", "fs.default.name":
> "file:///", "hive.metastore.sasl.enabled": "false" }
>
> }
>
> Useful portion of my code:
>
> Connection conn = new Driver().connect("jdbc:drill:zk=local",null);
> Statement stmt = conn.createStatement();
> ResultSet rs = stmt.executeQuery("show databases");
> while (rs.next())
>
> { String SCHEMA_NAME = rs.getString("SCHEMA_NAME");
> System.out.println(SCHEMA_NAME); }
>
> Exception:
>
> java.sql.SQLException: Failure in starting embedded Drillbit:
> java.lang.RuntimeException: Unable to deserialize
> "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:109)
> at
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
> at
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at
> net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:78)
> at java.sql.DriverManager.getConnection(DriverManager.java:571)
> at java.sql.DriverManager.getConnection(DriverManager.java:187)
> at com.mkyong.App.main(App.java:28)
> Caused by: java.lang.RuntimeException: Unable to deserialize
> "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> at
> org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:140)
> at
> org.apache.drill.exec.store.sys.local.FilePStore$Iter$DeferredEntry.getValue(FilePStore.java:219)
> at
> org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:168)
> at
> org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:132)
> at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:244)
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
> ... 7 more
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not
> resolve type id 'hive' into a subtype of [simple type, class
> org.apache.drill.common.logical.StoragePluginConfig]
> at [Source: [B@c64ff75; line: 2, column: 3]
> at
> com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
> at
> com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:849)
> at
> com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167)
> at
> com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)
> at
> com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
> at
> com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
> at
> com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41)
> at
> com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1269)
> at
> com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:912)
> at
> org.apache.drill.exec.store.sys.serialize.JacksonSerializer.deserialize(JacksonSerializer.java:44)
> at
> org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:138)
> ... 12 more
>
>
> ?
>
>
> Regards,
> Devender
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>