[jira] [Comment Edited] (CALCITE-2421) RexSimplify#simplifyAnds foregoes some simplications if unknownAsFalse set to true

["Zoltan Haindrich (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770930#comment-16770930 ] 

Zoltan Haindrich edited comment on CALCITE-2421 at 2/28/19 3:58 PM:
--------------------------------------------------------------------

this right now is only working for {{UnknownAs.FALSE}} - and only when the type is not nullable. [link to code|https://github.com/apache/calcite/blob/ef9f926061de21ad713a76ec3ec8110e5cbd92bf/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L306]

we could instead of that could generally rewrite
* {{A=A}}   to {{A IS NOT NULL OR NULL}} (similarily <=,>=)
* {{A!=A}} to {{A IS NULL AND NULL}} (similarily <,>)

With these and the existing unknownAs logics ; this could became true/false A IS [NOT] NULL correctly ; but when those cases does not apply it would in the worst case cost one more RexNodes:
* {{w(A=A) = 2w(A) +1 }}
* {{w(A IS NOT NULL OR NULL)=w(A) + 3}}
* if we consider that {{w(A)>=1}} we are at worst added one more nodes

I think {{IS [NOT] NULL}} could help stat estimators is there are some info about null counts


was (Author: kgyrtkirk):
this right now is only working for {{UnknownAs.FALSE}} - and only when the type is not nullable. [link to code|https://github.com/apache/calcite/blob/ef9f926061de21ad713a76ec3ec8110e5cbd92bf/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L306]

we could instead of that could generally rewrite
* {{A=A}}   to {{A IS NOT NULL OR NULL}} (similarily <=,>=)
* {{A!=A}} to {{A IS NULL AND NULL}} (similarily <,>)

With these and the existing unknownAs logics ; this could became true/false A IS [NOT] NULL correctly ; but when those cases does not apply it would at worst retain the same number of RexNodes (I was afraid that it would raise it):
* {{w(A=A) = 2w(A) +1 }}
* {{w(A IS NOT NULL OR NULL)=w(A) + 2}}
* if we consider that {{w(A)>=1}} we are at worst at the same node count

> RexSimplify#simplifyAnds foregoes some simplications if unknownAsFalse set to true
> ----------------------------------------------------------------------------------
>
>                 Key: CALCITE-2421
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2421
>             Project: Calcite
>          Issue Type: Bug
>            Reporter: Laurent Goujon
>            Assignee: Laurent Goujon
>            Priority: Minor
>              Labels: pull-request-available
>
> It looks like {{RexSimplify#simplifyAnds}} foregoes some comparison simplifications if {{unknownAsFalse}} is set to true, like {{A = A AND B = B}} which might be simplified to {{A IS NOT NULL AND B IS NOT NULL}} or even {{true}} if {{A}} and {{B}} are known to be not nullable.
> One consequence of this is that the selectivity value might be off as a {{=}} comparison has a selectivity of 15% whereas {{IS NOT NULL}} has a selectivity of 90%.
> The simplication is skipped because {{RexSimplify#simplifyList}} simplify all terms with {{unknownAsFalse}} set to {{false}}, but in {{RexSimplify#simplifyAnd2ForUnknownAsFalse}}, there's no attempt at trying again to simplify each term.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)