[jira] [Commented] (HIVE-28667) Starting HS2 will occur warn log: rules.HiveRelFieldTrimmer: Error initializing field trimmer instance

["Stamatis Zampetakis (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-28667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17980020#comment-17980020 ] 

Stamatis Zampetakis commented on HIVE-28667:
--------------------------------------------

RelFieldTrimmer class is a reflection based visitor that tries to find the appropriate method based on the type (Class) of the argument. The errors occurs while trying to register a method dispatcher for {{JdbcSort.class}} since there are multiple candidate methods that qualify:
 * trimFields(JdbcRel rel, ImmutableBitSet fieldsUsed, Set<RelDataTypeField> extraFields)
 * trimFields(Sort sort, ImmutableBitSet fieldsUsed, Set<RelDataTypeField> extraFields)

Given that there is no inheritance relationship between `Sort` and `JdbcRel` it is unclear which method should be used to handle JdbcSort leading to an exception that informs the user/dev about the ambiguous semantics.

It seems that the intention of adding trimFields(JdbcRel...) method was to have a fallback for Jdbc operators that are not handled explicitly by the trimmer but this can never work and just creates ambiguity. The reason that we get this error only during startup of HS2 and not during compilation is because in the current code it is impossible to call the RelFieldTrimmer code with Jdbc operators other than HiveJdbcConverter and JdbcHiveTableScan.

> Starting HS2 will occur warn log: rules.HiveRelFieldTrimmer: Error initializing field trimmer instance
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-28667
>                 URL: https://issues.apache.org/jira/browse/HIVE-28667
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Butao Zhang
>            Assignee: Stamatis Zampetakis
>            Priority: Major
>
> Tested with laster master branch:  commit id: *4898d7bfaa1437d39d25070e0ab63c5e0e0de9d7*
> Starting HS2 will see the following warn log. But HS2 would start successfully at the end.
>  
> {code:java}
> 2024-12-13T16:49:13,207  WARN [main] rules.HiveRelFieldTrimmer: Error initializing field trimmer instance
> java.lang.IllegalArgumentException: dispatch ambiguity between public org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer$TrimResult org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Sort,org.apache.calcite.util.ImmutableBitSet,java.util.Set) and public org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer$TrimResult org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer.trimFields(org.apache.calcite.adapter.jdbc.JdbcRel,org.apache.calcite.util.ImmutableBitSet,java.util.Set)
>         at org.apache.calcite.util.ReflectUtil.lookupVisitMethod(ReflectUtil.java:391) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.calcite.util.ReflectUtil.lookupVisitMethod(ReflectUtil.java:325) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReflectUtil$VisitDispatcher.lookupVisitFunc(HiveReflectUtil.java:165) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReflectUtil$HiveMethodDispatcher.register(HiveReflectUtil.java:255) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.register(HiveRelFieldTrimmer.java:871) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.initializeFieldTrimmerClass(HiveRelFieldTrimmer.java:880) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.warmup(CalcitePlanner.java:5605) ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:311) ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1190) ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1551) ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1358) ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
>         at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
>         at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
>         at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
>         at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
>         at org.apache.hadoop.util.RunJar.run(RunJar.java:328) ~[hadoop-common-3.3.6.jar:?]
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:241) ~[hadoop-common-3.3.6.jar:?]
>  {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)