Calcite - add new field to Project via RelOptRule

[Oleksandr Fedorov <ol...@oath.com.INVALID>]

Hello,

There is a overall task to pushdown an expression from outer select to
nested project (if it has all required dependent fields).

For replace purpose were selected RelOptRule, but when I try to replace
project with a newProject that has on 1 expression more - validation fails:

java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(DOUBLE field1) NOT NULL
expression type is RecordType(DOUBLE field1, DOUBLE newField) NOT NULL
set is
rel#135:LogicalProject.NONE.[](input=HepRelVertex#134,field1=$4,newField=$7)
expression is LogicalProject#187
    at
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:411)
    at
org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
    at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)

Rule:
    override def onMatch(call: RelOptRuleCall): Unit = {
      val project = _getProject(call)
      logger.info(s"onMatch() DO UPDATE, call=$call, project=$project")

      val rexCall = _formulaContext.formula.rexCall
      val builder = call.builder()

      val newRexNodes = new java.util.ArrayList[RexNode]()
      newRexNodes.addAll(project.getChildExps)
      newRexNodes.add(rexCall)
      val newRexNodeNames =
Seq[String](project.getRowType.getFieldNames.asScala :+
_formulaContext.formula.field: _*)

      val typeBuilder = call.builder().getTypeFactory.builder()
      typeBuilder.addAll(project.getRowType.getFieldList.asScala.map(f =>
new util.AbstractMap.SimpleEntry(f.getName, f.getType)).asJava)
      typeBuilder.add(_formulaContext.formula.field,
_formulaContext.formula.rexCall.`type`)

      val newProject = project.copy(project.getTraitSet, project.getInput,
newRexNodes, typeBuilder.build())

      call.transformTo(newProject)
    }


What is the correct way to update project by adding new expression to it ?

Thanks.

Re: Calcite - add new field to Project via RelOptRule

[Michael Mior <mm...@apache.org>]

If you want transformations which actually change the output of the
query, you probably just want to change the RelNode directly.
RelOptRules are expected to preserve the semantics of the query so you
shouldn't do things such as adding new fields or changing the
collation. You can still run the optimizer after you make these kinds
of changes.

--
Michael Mior
mmior@apache.org
Le jeu. 26 juil. 2018 à 08:57, Oleksandr Fedorov
<ol...@oath.com.invalid> a écrit :
>
> Hello,
>
> There is a overall task to pushdown an expression from outer select to
> nested project (if it has all required dependent fields).
>
> For replace purpose were selected RelOptRule, but when I try to replace
> project with a newProject that has on 1 expression more - validation fails:
>
> java.lang.AssertionError: Cannot add expression of different type to set:
> set type is RecordType(DOUBLE field1) NOT NULL
> expression type is RecordType(DOUBLE field1, DOUBLE newField) NOT NULL
> set is
> rel#135:LogicalProject.NONE.[](input=HepRelVertex#134,field1=$4,newField=$7)
> expression is LogicalProject#187
>     at
> org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:411)
>     at
> org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
>     at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
>
> Rule:
>     override def onMatch(call: RelOptRuleCall): Unit = {
>       val project = _getProject(call)
>       logger.info(s"onMatch() DO UPDATE, call=$call, project=$project")
>
>       val rexCall = _formulaContext.formula.rexCall
>       val builder = call.builder()
>
>       val newRexNodes = new java.util.ArrayList[RexNode]()
>       newRexNodes.addAll(project.getChildExps)
>       newRexNodes.add(rexCall)
>       val newRexNodeNames =
> Seq[String](project.getRowType.getFieldNames.asScala :+
> _formulaContext.formula.field: _*)
>
>       val typeBuilder = call.builder().getTypeFactory.builder()
>       typeBuilder.addAll(project.getRowType.getFieldList.asScala.map(f =>
> new util.AbstractMap.SimpleEntry(f.getName, f.getType)).asJava)
>       typeBuilder.add(_formulaContext.formula.field,
> _formulaContext.formula.rexCall.`type`)
>
>       val newProject = project.copy(project.getTraitSet, project.getInput,
> newRexNodes, typeBuilder.build())
>
>       call.transformTo(newProject)
>     }
>
>
> What is the correct way to update project by adding new expression to it ?
>
> Thanks.