[jira] [Created] (CALCITE-7876) VolcanoPlanner fires logical transformation rules such as ProjectMergeRule on JDBC rels

["Vladimir Sitnikov (Jira)" <ji...@apache.org>]

Vladimir Sitnikov created CALCITE-7876:
------------------------------------------

             Summary: VolcanoPlanner fires logical transformation rules such as ProjectMergeRule on JDBC rels
                 Key: CALCITE-7876
                 URL: https://issues.apache.org/jira/browse/CALCITE-7876
             Project: Calcite
          Issue Type: Bug
          Components: core, jdbc-adapter
            Reporter: Vladimir Sitnikov


When I plan a query against a JDBC schema, {{VolcanoPlanner}} fires logical transformation rules on JDBC rels. {{ProjectMergeRule}} matches two {{JdbcProject}} rels and produces a {{LogicalProject}}; {{FilterProjectTransposeRule}} matches {{JdbcFilter}} over {{JdbcProject}} and produces a {{JdbcProject}}. The same rules do not fire on Enumerable rels.

*Expected*

The javadoc of {{TransformationRule}} says: "only logical operator can be rule operand, and only generate logical alternatives". So in {{VolcanoPlanner}}, {{ProjectMergeRule}}, {{FilterProjectTransposeRule}}, and {{SortRemoveConstantKeysRule}} (via {{SubstitutionRule}}) should have no attempts whose first operand is a {{JdbcRel}}, which is what happens for {{EnumerableRel}} today.

*Actual*

Run on main at 6ff13eaccd (1.43.0-SNAPSHOT), with the {{scott}} JDBC model from the test suite. Rule attempts whose first operand is a {{JdbcRel}}, with the operand classes:

{noformat}
JDBC_ATTEMPTS={EnumerableLimitRule on JdbcSort,=2, FilterProjectTransposeRule on JdbcFilter,JdbcProject,=1, JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcFilter,=2, JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcProject,=4, JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcSort,=2, JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcTableScan,=1, ProjectMergeRule on JdbcProject,JdbcProject,=1, SortRemoveConstantKeysRule on JdbcSort,=2, SortRemoveRule on JdbcSort,=2}
JDBC_PRODUCED={EnumerableLimitRule -> EnumerableLimit=1, FilterProjectTransposeRule -> JdbcProject=1, JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) -> JdbcToEnumerableConverter=9, ProjectMergeRule -> LogicalProject=1, SortRemoveRule -> RelSubset=1}
{noformat}

The same probe on the {{hr}} reflective schema (Enumerable only) recorded one attempt on an {{EnumerableRel}}: {{EnumerableLimitRule}} on {{EnumerableSort}}, whose {{onMatch}} returns at once because the sort has no fetch.

I did not measure planning time, so I do not know how much these attempts cost on large queries.

*Reproducer* (a JUnit 5 test in {{core/src/test/java}})

{code:java}
@Test void transformationRulesOnJdbcRels() {
  Map<String, Integer> attempts = new TreeMap<>();
  Map<String, Integer> produced = new TreeMap<>();
  CalciteAssert.model(JdbcTest.SCOTT_MODEL)
      .query("select x + 1 as y from (select empno + deptno as x, ename"
          + " from scott.emp where deptno > 10) order by y limit 2")
      .withHook(Hook.PLANNER, (Consumer<RelOptPlanner>) planner ->
          planner.addListener(new RelOptListener() {
            @Override public void ruleAttempted(RuleAttemptedEvent e) {
              if (e.isBefore() && e.getRuleCall().rels[0] instanceof JdbcRel) {
                StringBuilder sb = new StringBuilder();
                for (RelNode r : e.getRuleCall().rels) {
                  sb.append(r.getClass().getSimpleName()).append(',');
                }
                attempts.merge(e.getRuleCall().getRule() + " on " + sb, 1,
                    Integer::sum);
              }
            }
            @Override public void ruleProductionSucceeded(RuleProductionEvent e) {
              if (!e.isBefore() && e.getRuleCall().rels[0] instanceof JdbcRel) {
                produced.merge(e.getRuleCall().getRule() + " -> "
                    + e.getRel().getClass().getSimpleName(), 1, Integer::sum);
              }
            }
            @Override public void relEquivalenceFound(RelEquivalenceEvent e) { }
            @Override public void relDiscarded(RelDiscardedEvent e) { }
            @Override public void relChosen(RelChosenEvent e) { }
          }))
      .returnsCount(2);
  System.out.println("JDBC_ATTEMPTS=" + attempts);
  System.out.println("JDBC_PRODUCED=" + produced);
}
{code}

*Analysis (a guess; the report above stands without it)*

{{VolcanoPlanner}} skips {{TransformationRule}} operands only for classes that implement {{PhysicalNode}} ({{VolcanoPlanner#addRule}} and {{VolcanoPlanner#onNewClass}}, added by CALCITE-3997). {{EnumerableRel}} extends {{PhysicalNode}}; {{JdbcRel}} and {{BindableRel}} extend {{RelNode}} only, and so do the rel interfaces of the other adapters. One possible fix is to make those interfaces extend {{PhysicalNode}}; I have not checked what else that changes, for example how {{passThrough}} and {{derive}} behave in top-down mode.

CALCITE-3455 reported this for Enumerable rels, and CALCITE-3997 fixed it there; this issue covers the conventions that CALCITE-3997 left out.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)