[jira] [Comment Edited] (HIVE-16797) Enhance HiveFilterSetOpTransposeRule to remove union branches

["Pengcheng Xiong (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-16797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037265#comment-16037265 ] 

Pengcheng Xiong edited comment on HIVE-16797 at 6/5/17 5:42 PM:
----------------------------------------------------------------

I use pull-up-constant to pull the constant out of union. Then use RexSimplify to simplify that to see if it can be reduced to always false. However, there are still several comments regarding patch 02: (1) it sounds like that I was not able to simplify (($2=1 OR $2=2) AND $2=3) to a false. Here ($2=1 OR $2=2) comes from two branches of union. Thus, I introduce a hive union merge rule (btw, calcite union merge rule does not fire well in current hive master) so that we can check  ($2=1 AND $2=3) and ($2=2 AND $2=3), repectively, which works with RexSimplify. (2) it sounds like RexSimplify also can not reduce ($2>2 AND $2=3) to false. There is a test case in filter_union.q for that and you will see. (3) if we can assume that it is always a  project under union, we may have better options. (4) for tpcds queries, current patch is good enough.


was (Author: pxiong):
I use pull-up-constant to pull the constant out of union. Then use RexSimplify to simplify that to see if it can be reduced to always false. However, there are still several comments regarding patch 02: (1) it sounds like that I was not able to simplify (($2=1 OR $2=2) AND $2=3) to a false. Here ($2=1 OR $2=2) comes from two branches of union. Thus, I introduce a hive union merge rule (btw, calcite union merge rule does not fire well in current hive master) so that we can check  ($2=1 AND $2=3) and ($2=2 AND $2=3), repectively, which works with RexSimplify. (2) it sounds like RexSimplify also can not reduce ($2>2 AND $2=3) to false. There is a test case in filter_union.q for that and you will see. (3) if we can assume that it is always a  project under union, we may have better options.

> Enhance HiveFilterSetOpTransposeRule to remove union branches
> -------------------------------------------------------------
>
>                 Key: HIVE-16797
>                 URL: https://issues.apache.org/jira/browse/HIVE-16797
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Pengcheng Xiong
>            Assignee: Pengcheng Xiong
>         Attachments: HIVE-16797.01.patch, HIVE-16797.02.patch
>
>
> in query4.q, we can see that it creates a CTE with union all of 3 branches. Then it is going to do a 3 way self-join of the CTE with predicates. The predicates actually specifies only one of the branch in CTE to participate in the join. Thus, in some cases, e.g.,
> {code}
>                /- filter(false) -----TS0 
> union all  - filter(false) -----TS1
>                \-----TS2
> {code}
> we can cut the branches of TS0 and TS1. The union becomes only TS2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)