[jira] [Commented] (HIVE-22126) hive-exec packaging should shade guava

["Chao Sun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17228795#comment-17228795 ] 

Chao Sun commented on HIVE-22126:
---------------------------------

Curious how does this work for interop with Calcite? we were trying to backport this to branch-2.3 and it failed with the following exception:

{code}
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
	... 48 more
Caused by: java.lang.NoSuchMethodError: org.apache.calcite.rel.RelCollationImpl.<init>(Lorg/apache/hive/com/google/common/collect/ImmutableList;)V
	at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelCollation.<init>(HiveRelCollation.java:29)
	at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getCollationList(RelOptHiveTable.java:181)
	at org.apache.calcite.rel.metadata.RelMdCollation.table(RelMdCollation.java:175)
	at org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:122)
	at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
	at GeneratedMetadataHandler_Collation.collations(Unknown Source)
	at org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:482)
	at org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:189)
	at org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
	at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:273)
	... 53 more
{code}

The problem is that {{HiveRelCollation}} is defined as follow:
{code}
public class HiveRelCollation extends RelCollationImpl {
  public HiveRelCollation(ImmutableList<RelFieldCollation> fieldCollations) {
    super(fieldCollations);
  }
}
{code}

while {{RelCollationImpl}} in Calcite is:
{code}
  protected RelCollationImpl(ImmutableList<RelFieldCollation> fieldCollations) {
    this.fieldCollations = fieldCollations;
    Preconditions.checkArgument(
        Util.isDistinct(RelCollations.ordinals(fieldCollations)),
        "fields must be distinct");
  }
{code}

So if we shade Guava as it is then Hive will use {{org/apache/hive/com/google/common/collect/ImmutableList}} in its API which is not available from the parent class? I'm not sure whether master branch has the same issue.

> hive-exec packaging should shade guava
> --------------------------------------
>
>                 Key: HIVE-22126
>                 URL: https://issues.apache.org/jira/browse/HIVE-22126
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Vihang Karajgaonkar
>            Assignee: Eugene Chung
>            Priority: Major
>             Fix For: 4.0.0
>
>         Attachments: HIVE-22126.01.patch, HIVE-22126.02.patch, HIVE-22126.03.patch, HIVE-22126.04.patch, HIVE-22126.05.patch, HIVE-22126.06.patch, HIVE-22126.07.patch, HIVE-22126.08.patch, HIVE-22126.09.patch, HIVE-22126.09.patch, HIVE-22126.09.patch, HIVE-22126.09.patch, HIVE-22126.09.patch
>
>
> The ql/pom.xml includes complete guava library into hive-exec.jar https://github.com/apache/hive/blob/master/ql/pom.xml#L990 This causes a problems for downstream clients of hive which have hive-exec.jar in their classpath since they are pinned to the same guava version as that of hive. 
> We should shade guava classes so that other components which depend on hive-exec can independently use a different version of guava as needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)