You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Hyde <jh...@apache.org> on 2015/05/18 20:51:21 UTC
Re: HIVE-9069
The methods that may help are RexUtil.toCnf and RexUtil.pullFactors.
The latter is not as strict, therefore does not cause a combinatorial
explosion in the size of the tree, so is more useful in practice. They are
tested in RexProgramTest.testCnf and testPullFactors[1] .
These methods canonize at the term level (i.e. “a or b or a” becomes “a or
b” but does not attempt to recognize terms that are equivalent). Neither of
them exploit the symmetry of ‘=‘ (i.e. a = b iff b = a). It would make
sense to normalize comparisons between field references & literals so that
the lower field reference is always on the left. So, "$6 = $3" becomes “$3
= $6”; “$6 > $3” becomes “$3 < $6”. And “literal <= $5” becomes “$5 >=
literal”. This would change a few test logs but would not damage
performance, and would improve a few plans.
Cab you please log a jira case with what you think would be useful even if
you don’t intend to write a patch.
Julian
[1]
https://github.com/apache/incubator-calcite/blob/master/core/src/test/java/org/apache/calcite/test/RexProgramTest.java#L562
On May 18, 2015, at 11:02 AM, Jesus Camachorodriguez <
jcamachorodriguez@hortonworks.com> wrote:
Julian,
This is the JIRA case for pushing the expressions from filters down:
https://issues.apache.org/jira/browse/HIVE-9069
<snip>
Let me know if you think a Calcite method could be applied to simplify
that predicate, or I could extend a method myself to cover those cases.
Thanks,
Jesús
Re: HIVE-9069
Posted by Jesus Camachorodriguez <jc...@hortonworks.com>.
I have logged CALCITE-739 to track this issue.
Jesús
On 5/19/15, 10:48 AM, "Jesus Camachorodriguez"
<jc...@hortonworks.com> wrote:
>Thanks Julian.
>
>Let me check what the methods that you pointed to can do.
>Once I check them, I will create a JIRA with a small enumeration of cases
>that could be covered by a possible extension to those methods.
>
>Jesús
>
>
>
>
>From: Julian Hyde <jh...@apache.org>
>Date: Monday, May 18, 2015 at 7:51 PM
>To: Jesus Camachorodriguez <jc...@hortonworks.com>
>Cc: "dev@calcite.incubator.apache.org" <de...@calcite.incubator.apache.org>
>Subject: Re: HIVE-9069
>
>
>The methods that may help are RexUtil.toCnf and RexUtil.pullFactors.
>
>The latter is not as strict, therefore does not cause a combinatorial
>explosion in the size of the tree, so is more useful in practice. They
>are tested in RexProgramTest.testCnf and testPullFactors[1] .
>
>These methods canonize at the term level (i.e. ³a or b or a² becomes ³a or
>b² but does not attempt to recognize terms that are equivalent). Neither
>of them exploit the symmetry of Œ=Œ (i.e. a = b iff b = a). It would make
>sense to normalize comparisons
> between field references & literals so that the lower field reference is
>always on the left. So, "$6 = $3" becomes ³$3 = $6²; ³$6 > $3² becomes ³$3
>< $6². And ³literal <= $5² becomes ³$5 >= literal². This would change a
>few test logs but would not damage performance,
> and would improve a few plans.
>
>Cab you please log a jira case with what you think would be useful even if
>you don¹t intend to write a patch.
>
>Julian
>
>[1]
>https://github.com/apache/incubator-calcite/blob/master/core/src/test/java
>/
>org/apache/calcite/test/RexProgramTest.java#L562
>
>On May 18, 2015, at 11:02 AM, Jesus Camachorodriguez
><jc...@hortonworks.com> wrote:
>
>
>Julian,
>
>This is the JIRA case for pushing the expressions from filters down:
>https://issues.apache.org/jira/browse/HIVE-9069
>
>
>
><snip>
>
>
>Let me know if you think a Calcite method could be applied to simplify
>that predicate, or I could extend a method myself to cover those cases.
>
>Thanks,
>Jesús
>
>
Re: HIVE-9069
Posted by Jesus Camachorodriguez <jc...@hortonworks.com>.
Thanks Julian.
Let me check what the methods that you pointed to can do.
Once I check them, I will create a JIRA with a small enumeration of cases
that could be covered by a possible extension to those methods.
Jesús
From: Julian Hyde <jh...@apache.org>
Date: Monday, May 18, 2015 at 7:51 PM
To: Jesus Camachorodriguez <jc...@hortonworks.com>
Cc: "dev@calcite.incubator.apache.org" <de...@calcite.incubator.apache.org>
Subject: Re: HIVE-9069
The methods that may help are RexUtil.toCnf and RexUtil.pullFactors.
The latter is not as strict, therefore does not cause a combinatorial
explosion in the size of the tree, so is more useful in practice. They
are tested in RexProgramTest.testCnf and testPullFactors[1] .
These methods canonize at the term level (i.e. ³a or b or a² becomes ³a or
b² but does not attempt to recognize terms that are equivalent). Neither
of them exploit the symmetry of Œ=Œ (i.e. a = b iff b = a). It would make
sense to normalize comparisons
between field references & literals so that the lower field reference is
always on the left. So, "$6 = $3" becomes ³$3 = $6²; ³$6 > $3² becomes ³$3
< $6². And ³literal <= $5² becomes ³$5 >= literal². This would change a
few test logs but would not damage performance,
and would improve a few plans.
Cab you please log a jira case with what you think would be useful even if
you don¹t intend to write a patch.
Julian
[1]
https://github.com/apache/incubator-calcite/blob/master/core/src/test/java/
org/apache/calcite/test/RexProgramTest.java#L562
On May 18, 2015, at 11:02 AM, Jesus Camachorodriguez
<jc...@hortonworks.com> wrote:
Julian,
This is the JIRA case for pushing the expressions from filters down:
https://issues.apache.org/jira/browse/HIVE-9069
<snip>
Let me know if you think a Calcite method could be applied to simplify
that predicate, or I could extend a method myself to cover those cases.
Thanks,
Jesús