[jira] [Commented] (HIVE-22126) hive-exec packaging should shade guava

["Stamatis Zampetakis (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503611#comment-17503611 ] 

Stamatis Zampetakis commented on HIVE-22126:
--------------------------------------------

As part of this change some transitive dependencies from calcite-core module, namely json-path, commons-compiler, and janino , have been declared explicitly in Hive (pom.xml). Previous comments mention briefly that this was necessary due to shading of calcite-core in ql module so this is an attempt to clarify a bit more the need for these explicit declarations.

The ql module shades "partially" some of its dependencies. Partially means that we shade *only* certain packages (e.g., {{org.apache.calcite:*}}) and not everything  (including transitive dependencies), which usually happens by default if we don't have specific {{<include>}} directives. If they are not declared explicitly the classes in transitive dependencies are not in the classpath thus CBO will fail at runtime.

The alternative would be to include the aforementioned transitive dependencies in the shaded jar by adding more include directives. Putting more things in the shaded jar increases its size and the chances for classpath problems when used by other projects. Moreover, as pointed out by Eugene previously, other kind of problems like jar signing would also occur so this is not really an option.

> hive-exec packaging should shade guava
> --------------------------------------
>
>                 Key: HIVE-22126
>                 URL: https://issues.apache.org/jira/browse/HIVE-22126
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Vihang Karajgaonkar
>            Assignee: Eugene Chung
>            Priority: Major
>             Fix For: 4.0.0
>
>         Attachments: HIVE-22126.01.patch, HIVE-22126.02.patch, HIVE-22126.03.patch, HIVE-22126.04.patch, HIVE-22126.05.patch, HIVE-22126.06.patch, HIVE-22126.07.patch, HIVE-22126.08.patch, HIVE-22126.09.patch, HIVE-22126.09.patch, HIVE-22126.09.patch, HIVE-22126.09.patch, HIVE-22126.09.patch
>
>
> The ql/pom.xml includes complete guava library into hive-exec.jar https://github.com/apache/hive/blob/master/ql/pom.xml#L990 This causes a problems for downstream clients of hive which have hive-exec.jar in their classpath since they are pinned to the same guava version as that of hive. 
> We should shade guava classes so that other components which depend on hive-exec can independently use a different version of guava as needed.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)