You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@impala.apache.org by Dinujaya Kaweesha <di...@gmail.com> on 2022/09/08 04:53:38 UTC
Hive metastore version to work with impala 4.1.0
Hi team,
I have recently built the impala 4.1.0 and started the daemons. There I
received "Invalid method name: 'get_database_req" error stamped in
catalogd while trying to connect to hive metastore. Hive metastore version
that I've used is Hive 3.1.3 (
https://github.com/apache/hive/tree/rel/release-3.1.3).
I was able to identify an issue ticket related to the above error but it is
supposed to be dispatched in hive 4.0.0 release which is not yet released(
HIVE-22087 <https://issues.apache.org/jira/browse/HIVE-22087>).
Could you please let me know whether there is any* open-source hive release*
currently available that supports *impala 4.1.0*?
Thanks & Regards,
Dinujaya Kaweesha
Re: Hive metastore version to work with impala 4.1.0
Posted by Quanlong Huang <hu...@gmail.com>.
I think you are hitting this issue on Java 9+:
https://issues.apache.org/jira/browse/HIVE-21508
It's fixed in Hive-3.1.3 but not in Hive-3.1.2, and unfortunately
Impala-4.1.0 still depends on Hive-3.1.2.
You can try these:
- Use Java 8 instead, i.e. compile and run Impala on Java 8.
- Apply the patch to bump the Apache Hive version to 3.1.3 and recompile
Impala: https://gerrit.cloudera.org/c/18716/
- Probably you just need to replace the Hive 3.1.2 jars with Hive
3.1.3 ones.
On Wed, Sep 14, 2022 at 12:13 PM Dinujaya Kaweesha <di...@gmail.com>
wrote:
> Hi Quanlong,
> Thank you for your reply.
>
> I have rebuilt the impala using the instructions you provided and build
> was successful.
> But catalogd terminates providing below error message.
> Do you have any update regarding this?
>
> E0913 21:30:02.936092 198642 MetaStoreUtils.java:166] Got exception:
> java.lang.ClassCastException class [Ljava.lang.Object; cannot be cast to
> class [Ljava.net.URI; ([Ljava.lang.Object; and [Ljava.net.URI; are in
> module java.base of loader 'bootstrap')
>
> Java exception follows:
>
> java.lang.ClassCastException: class [Ljava.lang.Object; cannot be cast to
> class [Ljava.net.URI; ([Ljava.lang.Object; and [Ljava.net.URI; are in
> module java.base of loader 'bootstrap')
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.resolveUris(HiveMetaStoreClient.java:262)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:182)
>
> at
> jdk.internal.reflect.GeneratedConstructorAccessor4.newInstance(Unknown
> Source)
>
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
>
> at
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:112)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:78)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:174)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:163)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:155)
>
> at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:144)
>
> E0913 21:30:02.936192 198642 MetaStoreUtils.java:167] Converting exception
> to MetaException
>
> W0913 21:30:02.936322 198642 MetaStoreClientPool.java:107] Failed to
> connect to Hive MetaStore. Retrying.
>
> Java exception follows:
>
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
>
> at
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:112)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
>
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClient
>
> Thanks & Regards,
> Dinujaya Kaweesha
>
> On Fri, 9 Sep 2022 at 11:04, Quanlong Huang <hu...@gmail.com>
> wrote:
>
>> Hi Dinujaya,
>>
>> Did you compile Impala with "export USE_APACHE_HIVE=true" ? You might
>> also need an additional patch: https://gerrit.cloudera.org/c/18028
>> (click the download button to get it).
>>
>> Thanks,
>> Quanlong
>>
>> On Thu, Sep 8, 2022 at 12:54 PM Dinujaya Kaweesha <di...@gmail.com>
>> wrote:
>>
>>> Hi team,
>>> I have recently built the impala 4.1.0 and started the daemons. There I
>>> received "Invalid method name: 'get_database_req" error stamped in
>>> catalogd while trying to connect to hive metastore. Hive metastore version
>>> that I've used is Hive 3.1.3 (
>>> https://github.com/apache/hive/tree/rel/release-3.1.3).
>>>
>>> I was able to identify an issue ticket related to the above error but it
>>> is supposed to be dispatched in hive 4.0.0 release which is not yet
>>> released(HIVE-22087 <https://issues.apache.org/jira/browse/HIVE-22087>
>>> ).
>>>
>>> Could you please let me know whether there is any* open-source hive
>>> release* currently available that supports *impala 4.1.0*?
>>>
>>> Thanks & Regards,
>>> Dinujaya Kaweesha
>>>
>>
Re: Hive metastore version to work with impala 4.1.0
Posted by Dinujaya Kaweesha <di...@gmail.com>.
Hi Quanlong,
Thank you for your reply.
I have rebuilt the impala using the instructions you provided and build was
successful.
But catalogd terminates providing below error message.
Do you have any update regarding this?
E0913 21:30:02.936092 198642 MetaStoreUtils.java:166] Got exception:
java.lang.ClassCastException class [Ljava.lang.Object; cannot be cast to
class [Ljava.net.URI; ([Ljava.lang.Object; and [Ljava.net.URI; are in
module java.base of loader 'bootstrap')
Java exception follows:
java.lang.ClassCastException: class [Ljava.lang.Object; cannot be cast to
class [Ljava.net.URI; ([Ljava.lang.Object; and [Ljava.net.URI; are in
module java.base of loader 'bootstrap')
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.resolveUris(HiveMetaStoreClient.java:262)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:182)
at
jdk.internal.reflect.GeneratedConstructorAccessor4.newInstance(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:112)
at
org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
at
org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:78)
at
org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:174)
at
org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:163)
at
org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:155)
at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:144)
E0913 21:30:02.936192 198642 MetaStoreUtils.java:167] Converting exception
to MetaException
W0913 21:30:02.936322 198642 MetaStoreClientPool.java:107] Failed to
connect to Hive MetaStore. Retrying.
Java exception follows:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at
org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:112)
at
org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
at
org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClient
Thanks & Regards,
Dinujaya Kaweesha
On Fri, 9 Sep 2022 at 11:04, Quanlong Huang <hu...@gmail.com> wrote:
> Hi Dinujaya,
>
> Did you compile Impala with "export USE_APACHE_HIVE=true" ? You might also
> need an additional patch: https://gerrit.cloudera.org/c/18028 (click the
> download button to get it).
>
> Thanks,
> Quanlong
>
> On Thu, Sep 8, 2022 at 12:54 PM Dinujaya Kaweesha <di...@gmail.com>
> wrote:
>
>> Hi team,
>> I have recently built the impala 4.1.0 and started the daemons. There I
>> received "Invalid method name: 'get_database_req" error stamped in
>> catalogd while trying to connect to hive metastore. Hive metastore version
>> that I've used is Hive 3.1.3 (
>> https://github.com/apache/hive/tree/rel/release-3.1.3).
>>
>> I was able to identify an issue ticket related to the above error but it
>> is supposed to be dispatched in hive 4.0.0 release which is not yet
>> released(HIVE-22087 <https://issues.apache.org/jira/browse/HIVE-22087>).
>>
>> Could you please let me know whether there is any* open-source hive
>> release* currently available that supports *impala 4.1.0*?
>>
>> Thanks & Regards,
>> Dinujaya Kaweesha
>>
>
Re: Hive metastore version to work with impala 4.1.0
Posted by Quanlong Huang <hu...@gmail.com>.
Hi Dinujaya,
Did you compile Impala with "export USE_APACHE_HIVE=true" ? You might also
need an additional patch: https://gerrit.cloudera.org/c/18028 (click the
download button to get it).
Thanks,
Quanlong
On Thu, Sep 8, 2022 at 12:54 PM Dinujaya Kaweesha <di...@gmail.com>
wrote:
> Hi team,
> I have recently built the impala 4.1.0 and started the daemons. There I
> received "Invalid method name: 'get_database_req" error stamped in
> catalogd while trying to connect to hive metastore. Hive metastore version
> that I've used is Hive 3.1.3 (
> https://github.com/apache/hive/tree/rel/release-3.1.3).
>
> I was able to identify an issue ticket related to the above error but it
> is supposed to be dispatched in hive 4.0.0 release which is not yet
> released(HIVE-22087 <https://issues.apache.org/jira/browse/HIVE-22087>).
>
> Could you please let me know whether there is any* open-source hive
> release* currently available that supports *impala 4.1.0*?
>
> Thanks & Regards,
> Dinujaya Kaweesha
>