[jira] [Comment Edited] (HIVE-23389) FilterMergeRule can lead to AssertionError

["Jesus Camacho Rodriguez (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-23389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104485#comment-17104485 ] 

Jesus Camacho Rodriguez edited comment on HIVE-23389 at 5/11/20, 2:34 PM:
--------------------------------------------------------------------------

Thanks [~kgyrtkirk], pushed to master.

I think the {{Bug}} class is a good idea to keep track of additional work to do when we upgrade Calcite (I am sure there are TODO items over the code that could have been solved by now). I agree that there is a missing piece that should trigger some error when the dependency is updated so a dev is forced to check the file (currently we are always the same people upgrading the Calcite dependency, but that may not always be the case). Not sure how we could do that though. If there are any ideas, we can create a follow-up to add the check.


was (Author: jcamachorodriguez):
Thanks [~kgyrtkirk], pushed to master.

I think the {{Bug}} class is a good idea to keep track of additional work to do when we upgrade Calcite (I am sure there are TODO items over the code that could have been solved by now). I agree that there is a missing piece that should trigger some error when the dependency is updated so a dev is forced to check the file (currently we are always the same people upgrading the Calcite dependency, that that may not always be the case). Not sure how we could do that though. If there are any ideas, we can create a follow-up to add the check.

> FilterMergeRule can lead to AssertionError
> ------------------------------------------
>
>                 Key: HIVE-23389
>                 URL: https://issues.apache.org/jira/browse/HIVE-23389
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>         Attachments: HIVE-23389.01.patch, HIVE-23389.01.patch, HIVE-23389.02.patch, HIVE-23389.patch
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> I have not been able to reproduce the issue in latest master. However, this could potentially happen since Filter creation has a check on whether the expression is flat ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74]) and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
> 	at org.apache.calcite.rel.core.Filter.<init>(Filter.java:74)
> 	at org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.<init>(HiveFilter.java:39)
> 	at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
> 	at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
> 	at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
> 	at org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)