[jira] [Created] (IMPALA-10906) Adjust dependencies after HIVE-24852

["Fang-Yu Rao (Jira)" <ji...@apache.org>]

Fang-Yu Rao created IMPALA-10906:
------------------------------------

             Summary: Adjust dependencies after HIVE-24852
                 Key: IMPALA-10906
                 URL: https://issues.apache.org/jira/browse/IMPALA-10906
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Fang-Yu Rao
            Assignee: Fang-Yu Rao


HIVE-24852 added the dependency on the artifact {{hadoop-hdfs}} under the module of {{hive-service}}, which is a dependency of {{hive-jdbc}} that Impala relies on. However, since {{hadoop-hdfs}} transitively pulls in the artifact {{jersey-server}}, which is a banned dependency by Impala, we had to explicitly exclude {{jersey-server}} when adding {{hive-jdbc}} as a dependency so that Impala frontend could be compiled next time when we bump up {{CDP_BUILD_NUMBER}} that includes this Hive patch.

Moreover, due to the fact that after HIVE-24852, the creation of a partitioned iceberg table requires the existence of the class {{org.apache.hadoop.hdfs.protocol.SnapshotException}} on Impala's classpath at runtime, we had to explicitly add the dependency on the artifact {{hadoop-hdfs}} so that such an operation will not result in a {{NoClassDefFoundError}}. We also need to  explicitly excluded some banned artifacts that were transitively pulled in by {{hadoop-hdfs}} so that Impala frontend could be compiled.

For easy reference, a query to reproduce the {{NoClassDefFoundError}} issue is provided in the following. A very similar query has been executed during the loading of Impala's test data so that once the dependencies are properly revised, we expect Impala to be able to load test data correctly.
{code:sql}
[localhost:21050] default> CREATE TABLE IF NOT EXISTS default.iceberg_int_partitioned_tbl (i INT, j INT, k INT)
                         > PARTITION BY SPEC (i identity, j identity)
                         > STORED AS ICEBERG;
Query: CREATE TABLE IF NOT EXISTS default.iceberg_int_partitioned_tbl (i INT, j INT, k INT)
PARTITION BY SPEC (i identity, j identity)
STORED AS ICEBERG
ERROR: NoClassDefFoundError: org/apache/hadoop/hdfs/protocol/SnapshotException
CAUSED BY: ClassNotFoundException: org.apache.hadoop.hdfs.protocol.SnapshotException
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org