You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Anup Tiwari <an...@games24x7.com> on 2018/02/12 15:53:00 UTC
Unable to setup hive plugin in Drill 1.11.0
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Re: Unable to setup hive plugin in Drill 1.11.0
Posted by Anup Tiwari <an...@games24x7.com>.
Hi Arjun,
You have pointed correctly; i have removed all hive-2.1 jar from 3rd party and
kept only 3 jars which come with tar ball and it worked.
Thanks!!
On Tue, Feb 13, 2018 1:33 PM, Arjun kr arjun.kr@outlook.com wrote:
Hi Anup,
As Sorabh mentioned, you seem to be using hive 2.1.1 jars in Drill classpath
based on the stack trace. Did you build drill package by customizing hive
version to 2.1.1 or added hive 2.1.1 jars in Drill classpath manually? I could
see that Drill 1.12 (latest released), 1.11 and 1.10 are bundled with hive 1.2.1
client.
You could try with drill 1.11 tarball that includes hive 1.2.1 client. Also, you
may check the version of hive client jars in $DRILL_HOME/jars/3rdparty/
directory from the Drill 1.10 cluster where it is working fine.
http://archive.apache.org/dist/drill/drill-1.11.0/
# List of hive client jars from Drill 1.11 tarball extracted.
$ ls -1 apache-drill-1.11.0/jars/3rdparty/hive-*
apache-drill-1.11.0/jars/3rdparty/hive-contrib-1.2.1.jar
apache-drill-1.11.0/jars/3rdparty/hive-hbase-handler-1.2.1.jar
apache-drill-1.11.0/jars/3rdparty/hive-metastore-1.2.1.jar
$
Thanks,
Arjun
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Tuesday, February 13, 2018 12:01 PM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of
clusters and it is working properly. Just the change is in Hadoop Version and we
are using MySQL community version 5.7.16. Please find below details :-
Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache
Hadoop : 2.7.3
So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1
related jars in 3rdparty jar of drill.
On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?
On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwasia@mapr.com wrote:
Hi Anup,
From the stack trace which you have shared it looks like you are trying to use
Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.
Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.
[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[https://avatars3.githubusercontent.com/u/47359?s=400&v=4]<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>
apache/drill<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>
github.com
drill - Mirror of Apache Drill
[2]: https://github.com/apache/drill/pull/1111
Thanks,
Sorabh
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Monday, February 12, 2018 9:21 AM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Sent with Mixmax
Re: Unable to setup hive plugin in Drill 1.11.0
Posted by Arjun kr <ar...@outlook.com>.
Hi Anup,
As Sorabh mentioned, you seem to be using hive 2.1.1 jars in Drill classpath based on the stack trace. Did you build drill package by customizing hive version to 2.1.1 or added hive 2.1.1 jars in Drill classpath manually? I could see that Drill 1.12 (latest released), 1.11 and 1.10 are bundled with hive 1.2.1 client.
You could try with drill 1.11 tarball that includes hive 1.2.1 client. Also, you may check the version of hive client jars in $DRILL_HOME/jars/3rdparty/ directory from the Drill 1.10 cluster where it is working fine.
http://archive.apache.org/dist/drill/drill-1.11.0/
# List of hive client jars from Drill 1.11 tarball extracted.
$ ls -1 apache-drill-1.11.0/jars/3rdparty/hive-*
apache-drill-1.11.0/jars/3rdparty/hive-contrib-1.2.1.jar
apache-drill-1.11.0/jars/3rdparty/hive-hbase-handler-1.2.1.jar
apache-drill-1.11.0/jars/3rdparty/hive-metastore-1.2.1.jar
$
Thanks,
Arjun
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Tuesday, February 13, 2018 12:01 PM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of
clusters and it is working properly. Just the change is in Hadoop Version and we
are using MySQL community version 5.7.16. Please find below details :-
Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache
Hadoop : 2.7.3
So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1
related jars in 3rdparty jar of drill.
On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?
On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwasia@mapr.com wrote:
Hi Anup,
From the stack trace which you have shared it looks like you are trying to use
Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.
Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.
[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[https://avatars3.githubusercontent.com/u/47359?s=400&v=4]<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>
apache/drill<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>
github.com
drill - Mirror of Apache Drill
[2]: https://github.com/apache/drill/pull/1111
Thanks,
Sorabh
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Monday, February 12, 2018 9:21 AM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Re: Unable to setup hive plugin in Drill 1.11.0
Posted by Anup Tiwari <an...@games24x7.com>.
Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of
clusters and it is working properly. Just the change is in Hadoop Version and we
are using MySQL community version 5.7.16. Please find below details :-
Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache
Hadoop : 2.7.3
So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1
related jars in 3rdparty jar of drill.
On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?
On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwasia@mapr.com wrote:
Hi Anup,
From the stack trace which you have shared it looks like you are trying to use
Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.
Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.
[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[2]: https://github.com/apache/drill/pull/1111
Thanks,
Sorabh
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Monday, February 12, 2018 9:21 AM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Re: Unable to setup hive plugin in Drill 1.11.0
Posted by Anup Tiwari <an...@games24x7.com>.
Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?
On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwasia@mapr.com wrote:
Hi Anup,
From the stack trace which you have shared it looks like you are trying to use
Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.
Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.
[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[2]: https://github.com/apache/drill/pull/1111
Thanks,
Sorabh
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Monday, February 12, 2018 9:21 AM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari
Regards,
Anup Tiwari
Re: Unable to setup hive plugin in Drill 1.11.0
Posted by Sorabh Hamirwasia <sh...@mapr.com>.
Hi Anup,
From the stack trace which you have shared it looks like you are trying to use Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a PR[2] open for it in 1.13 though.
Also from the stack trace it's pointing towards an invalid field in HiveConf METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
Usually any error/exxception should get logged into drillbit.out as well. May be you can try seeing there.
[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[2]: https://github.com/apache/drill/pull/1111
Thanks,
Sorabh
________________________________
From: Anup Tiwari <an...@games24x7.com>
Sent: Monday, February 12, 2018 9:21 AM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari
Re: Unable to setup hive plugin in Drill 1.11.0
Posted by Anup Tiwari <an...@games24x7.com>.
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiwari@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari