[jira] [Created] (IMPALA-11801) Catalogd failed to start when compiled with Apache Hive3

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-11801:
---------------------------------------

             Summary: Catalogd failed to start when compiled with Apache Hive3
                 Key: IMPALA-11801
                 URL: https://issues.apache.org/jira/browse/IMPALA-11801
             Project: IMPALA
          Issue Type: Bug
          Components: Frontend
    Affects Versions: Impala 4.2.0
            Reporter: Quanlong Huang


Impala can still compile on Apache Hive 3 by setting env var USE_APACHE_HIVE:
{code:java}
export USE_APACHE_HIVE=true
./buildall.sh -noclean -notests -ninja{code}
However, starting catalogd would fail by a ClassNotFoundException:
{noformat}
I1215 14:47:20.640384 22724 jni-util.cc:288] java.lang.NoClassDefFoundError: org/apache/thrift/transport/TFramedTransport
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.hadoop.hive.metastore.utils.JavaUtils.getClass(JavaUtils.java:52)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:146)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:112)
        at org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
        at org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:78)
        at org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:174)
        at org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:163)
        at org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:155)
        at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:148)
Caused by: java.lang.ClassNotFoundException: org.apache.thrift.transport.TFramedTransport
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 12 more
I1215 14:47:20.640403 22724 status.cc:129] NoClassDefFoundError: org/apache/thrift/transport/TFramedTransport
CAUSED BY: ClassNotFoundException: org.apache.thrift.transport.TFramedTransport
    @           0xf5d3b7  impala::Status::Status()
    @          0x1778346  impala::JniUtil::GetJniExceptionMsg()
    @           0xf28137  impala::Catalog::Catalog()
    @           0xeef4b8  impala::CatalogServer::Start()
    @           0xe5287d  CatalogdMain()
    @           0xe5142f  main
    @     0x7f47eb564c87  __libc_start_main
    @           0xe5126a  _start {noformat}
The package name of TFramedTransport has changed from "org/apache/thrift/transport" to "org/apache/thrift/transport/layered" in thrift-0.16.0. The HMS client of Apache Hive3 is compiled with thrift-0.9.3 so it can't find the class.

The related thrift change:
https://github.com/apache/thrift/commit/63213c17ad3fece91fdaaca8f59165ca3f41c5c1

We should probably upgrade the dependent Apache Hive version to 4.x, and let Impala-4.1.x supports Apache Hive3.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)