[jira] [Updated] (HIVE-25852) Introduce IN clauses at the very end of query planning

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HIVE-25852:
----------------------------------
    Labels: pull-request-available  (was: )

> Introduce IN clauses at the very end of query planning
> ------------------------------------------------------
>
>                 Key: HIVE-25852
>                 URL: https://issues.apache.org/jira/browse/HIVE-25852
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>    Affects Versions: 4.0.0
>            Reporter: Alessandro Solimando
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Calcite "explodes" IN clauses into the equivalent OR form, and therefore it does not handle such clauses in most of the codebase (notably in _RexSimplify_).
> In Hive, the same happens, but _HivePointLookupOptimizerRule_ re-introduces IN clauses, and it happens in _applyPreJoinOrderingTransforms_ phase, which is pretty early and which mixes several other rules which might not fully support IN (notably, _HiveReduceExpressionsRule_ which is based on _RexSimplify_).
> The problem will become even harder in later versions of Calcite (current is 1.25) based on SARG, which does not support IN clauses.
> IN clauses can be converted into efficient runtime operators, we therefore want to keep them in the final plan, intuitively we just want this translation to happen in a later step, in order to leave the rest of the codebase (Hive and Calcite) unaware of IN clauses.
> The goal of the ticket is as follows:
> # re-convert the output expression of _HivePointLookupOptimizerRule_ into the OR form (keep the logic as-is to benefit from the rule)
> # add a rule, in the last step of the planning process, that only converts eligible OR expressions into IN clauses



--
This message was sent by Atlassian Jira
(v8.20.1#820001)