You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Mihai Budiu (Jira)" <ji...@apache.org> on 2024/10/23 03:38:00 UTC
[jira] [Commented] (CALCITE-6639) Optimization that pulls up predicates causes ASOF join validation failures
[ https://issues.apache.org/jira/browse/CALCITE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17892036#comment-17892036 ]
Mihai Budiu commented on CALCITE-6639:
--------------------------------------
Please note that this problem surfaced only after recent improvements to the optimization that pulls up predicate, most likely [CALCITE-6599] s; it didn't exist when the ASOF join was merged into main.
> Optimization that pulls up predicates causes ASOF join validation failures
> --------------------------------------------------------------------------
>
> Key: CALCITE-6639
> URL: https://issues.apache.org/jira/browse/CALCITE-6639
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.38.0
> Reporter: Mihai Budiu
> Assignee: Mihai Budiu
> Priority: Minor
>
> The following test added to the RelOptRulesTest causes an assertion failure:
> {code:java}
> @Test void testAsofOpt() {
> final String sql = "SELECT *\n" +
> "FROM (VALUES (NULL, 0), (1, NULL), (1, 0), (1, 1), (1, 2), (1, 3), (1, 4), (2, 3), (3, 4)) AS t1(k, t)\n" +
> "ASOF JOIN (VALUES (1, NULL), (1, 2), (1, 3), (2, 10), (2, 0)) AS t2(k, t)\n" +
> "MATCH_CONDITION t2.t < t1.t\n" +
> "ON t1.k = t2.k\n";
> sql(sql).withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS)
> .checkUnchanged();
> }
> {code}
> This test is taken straigt from asof.iq.
> This test fails with the following assertion failure:
> {code}
> java.lang.AssertionError
> at org.apache.calcite.rel.metadata.RelMdPredicates$JoinConditionBasedPredicateInference.inferPredicates(RelMdPredicates.java:850)
> at org.apache.calcite.rel.metadata.RelMdPredicates.getPredicates(RelMdPredicates.java:346)
> at org.apache.calcite.rel.metadata.janino.GeneratedMetadata_PredicatesHandler.getPredicates_$(Unknown Source)
> at org.apache.calcite.rel.metadata.janino.GeneratedMetadata_PredicatesHandler.getPredicates(Unknown Source)
> at org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:887)
> at org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:310)
> {code}
> Diagnosis: the definition of ASOF join implementation, inspired by the Snowflake variant of the operation, imposes some strong "syntactical" constraints on the predicate structure. In particular, each predicate in the equi-join part must compare expressions from the two joined relations.
> These constraints ensure that the ASOF join can be efficiently implemented. However, this definition inhibits some optimizations, such as pull-up predicates, which may rewrite predicates into a shape that does not satisfy the above rules.
> It is not clear what the right course of action is.
> One solution would be to relax the requirements on the ASOF join and to validate its shape only after optimizations. But then it is not clear that the resulting program would always be efficiently implementable - the original goal of ASOF.
> Another solution would be to inhibit optimizations from optimizing such joins.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)