[jira] [Comment Edited] (CALCITE-2615) When simplifying NOT-AND-OR, RexSimplify incorrectly applies predicates deduced for operands to the same operands

["Zoltan Haindrich (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642472#comment-16642472 ] 

Zoltan Haindrich edited comment on CALCITE-2615 at 10/8/18 9:26 PM:
--------------------------------------------------------------------

I've found a smaller sample:

{code}
  @Test
  public void testSimplifyNotAnd3() {
    final RexNode e = or(
        le(
            vBool(1),
            literal(true)),
        eq(
            literal(false),
            eq(literal(false), vBool(1))));
    final String expected = "TODO";
    checkSimplify(e, expected);
  }
{code}

the following is happening:

* simpifyOrTerms adds predicate {{not(vBool1 <= true)}} => {{(vBool1 > true)}} 
* I think the main problem is that the presence of this predicate also implies that {{vBool1 is NOT NULL}}
* the comparision {{=(false, vBool(1))}} is evaluated to false; because of range checks in simplifyUsingPredicates
* {{=(false, vBool1)}} becames false
** {{(vBool1 <= true) or (false = false)}}
* rest of the OR is discarded 

I think the main problem is that simplifyUsingPredicates exploited that vBool1 may not be null.

I'm not sure why; but the unknownAs guard is missing for simplifyOrTerms call.


was (Author: kgyrtkirk):
I've found a smaller sample:

{code}
  @Test
  public void testSimplifyNotAnd3() {
    final RexNode e = or(
        le(
            vBool(1),
            literal(true)),
        eq(
            literal(false),
            eq(literal(false), vBool(1))));
    final String expected = "TODO";
    checkSimplify(e, expected);
  }
{code}

the following is happening:

* simpifyOrTerms adds predicate {{not(vBool1 <= true)}} => {{(vBool1 > true)}} 
* I think the main problem is that the presence of this predicate also implies that {{vBool1 is NOT NULL}}
* the comparision {{=(false, vBool(1))}} is evaluated to false; because of range checks in simplifyUsingPredicates
* {{=(false, vBool1)}} becames false
** { (vBool1 <= true) or (false = false) }}
* rest of the OR is discarded 

I think the main problem is that simplifyUsingPredicates exploited that vBool1 may not be null.

I'm not sure why; but the unknownAs guard is missing for simplifyOrTerms call.

> When simplifying NOT-AND-OR, RexSimplify incorrectly applies predicates deduced for operands to the same operands
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: CALCITE-2615
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2615
>             Project: Calcite
>          Issue Type: Task
>            Reporter: Julian Hyde
>            Assignee: Julian Hyde
>            Priority: Major
>              Labels: newbie
>
> When simplifying NOT-AND-OR, RexSimplify incorrectly applies predicates deduced for operands to the same operands.
> Here is the test case (add it to RexProgramTest):
> {code:java}
>   @Test public void testSimplifyNotAnd() {
>     final RexNode e = not(
>         and(
>             gt(and(vBool(1), literal(true)),
>                 or(literal(true), literal(true), literal(false))),
>             gt(isNotNull(vBool(0)), eq(literal(false), vBool(1))),
>             or(ne(literal(true), literal(false)),
>                 ge(vInt(0), literal((Integer) null)))));
>     final String expected = "TODO";
>    checkSimplify(e, expected);
>   }
> {code}
> When you run it, verify will find a combination of assignments such that the simplified expression returns a different result than the original.
> The test case is not minimal; sorry. Maybe it reproduces with NOT-OR.
> The bug is in [RexSimplify.simplifyAndTerms|https://github.com/apache/calcite/blob/6b3844c0634792263a5073b8ea93565fb3415f41/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L412]. Put a breakpoint at that line to see the problem. It passes through the operands once, building a list of predicates. Then it passes through the operands again, simplifying each operand. Thus operand1 is simplified using a list of predicates that includes the predicate 'not operand1'. Clearly wrong. [~kgyrtkirk], I warned that this was possible in our discussions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)