You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Yang Liu <wh...@gmail.com> on 2020/03/05 07:48:01 UTC
rewrite an optimized RelNode
In Calcite, after optimization provided by the default VolcanoPlanner, we can get an optimized RelNode, but can we have a further optimization? For example I want to add an ElasticsearchSort or something like that to limit the dataset we will handle.
Someone suggests we can define a RelOptRule, but since VolcanoPlanner handles the optimization in dynamic programming way, not sure if the rule can be applied in the right order.
Any ideas are welcome~
Re: rewrite an optimized RelNode
Posted by Stamatis Zampetakis <za...@gmail.com>.
Hi Yang,
You can perform further transformations using the heuristic planner [1],
the RelVisitor [2], or RelShuttle [3].
Best,
Stamatis
[1]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/hep/HepPlanner.java
[2]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/RelVisitor.java
[3]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/RelShuttle.java
On Thu, Mar 5, 2020 at 5:24 PM Yang Liu <wh...@gmail.com> wrote:
> I write following code to get the optimized RelNode
>
> <code>
> val configBuilder = Frameworks.newConfigBuilder()
> configBuilder.defaultSchema(rootSchema)
> val frameworkConfig = configBuilder.build()
>
> val planner = Frameworks.getPlanner(frameworkConfig)
>
> val sql2 = """ select * from "insight_user"."user_tab" as t1 join
> "view"."logic_table_1" as t2 on t1."email" = t2."user" where t2."status" =
> 'fail' limit 10 """
> val sqlNode = planner.parse(sql2)
> val validate = planner.validate(sqlNode)
> val rel = planner.rel(validate).project()
>
> val cluster = rel.getCluster
> val optPlanner = cluster.getPlanner.asInstanceOf[VolcanoPlanner]
> // println(optPlanner.getRules)
> optPlanner.addRule(FilterJoinRule.FILTER_ON_JOIN)
> val desiredTraits =
> cluster.traitSet().replace(EnumerableConvention.INSTANCE)
> val mid = optPlanner.changeTraits(rel, desiredTraits)
> optPlanner.setRoot(mid)
> val optimized = optPlanner.findBestExp()
> </code>
>
> Now I want to rewrite the `optimized` RelNode.
>
>
>
>
> Danny Chan <yu...@gmail.com> 于2020年3月5日周四 下午10:33写道：
>
> > Is your optimize an implementation ? Or heuristic or needs some metadata?
> >
> > Yang Liu <wh...@gmail.com>于2020年3月5日 周四下午3:48写道：
> >
> > > In Calcite, after optimization provided by the default VolcanoPlanner,
> we
> > > can get an optimized RelNode, but can we have a further optimization?
> For
> > > example I want to add an ElasticsearchSort or something like that to
> > limit
> > > the dataset we will handle.
> > >
> > > Someone suggests we can define a RelOptRule, but since VolcanoPlanner
> > > handles the optimization in dynamic programming way, not sure if the
> rule
> > > can be applied in the right order.
> > >
> > > Any ideas are welcome~
> > >
> >
>
Re: rewrite an optimized RelNode
Posted by Yang Liu <wh...@gmail.com>.
I write following code to get the optimized RelNode
<code>
val configBuilder = Frameworks.newConfigBuilder()
configBuilder.defaultSchema(rootSchema)
val frameworkConfig = configBuilder.build()
val planner = Frameworks.getPlanner(frameworkConfig)
val sql2 = """ select * from "insight_user"."user_tab" as t1 join
"view"."logic_table_1" as t2 on t1."email" = t2."user" where t2."status" =
'fail' limit 10 """
val sqlNode = planner.parse(sql2)
val validate = planner.validate(sqlNode)
val rel = planner.rel(validate).project()
val cluster = rel.getCluster
val optPlanner = cluster.getPlanner.asInstanceOf[VolcanoPlanner]
// println(optPlanner.getRules)
optPlanner.addRule(FilterJoinRule.FILTER_ON_JOIN)
val desiredTraits =
cluster.traitSet().replace(EnumerableConvention.INSTANCE)
val mid = optPlanner.changeTraits(rel, desiredTraits)
optPlanner.setRoot(mid)
val optimized = optPlanner.findBestExp()
</code>
Now I want to rewrite the `optimized` RelNode.
Danny Chan <yu...@gmail.com> 于2020年3月5日周四 下午10:33写道：
> Is your optimize an implementation ? Or heuristic or needs some metadata?
>
> Yang Liu <wh...@gmail.com>于2020年3月5日 周四下午3:48写道：
>
> > In Calcite, after optimization provided by the default VolcanoPlanner, we
> > can get an optimized RelNode, but can we have a further optimization? For
> > example I want to add an ElasticsearchSort or something like that to
> limit
> > the dataset we will handle.
> >
> > Someone suggests we can define a RelOptRule, but since VolcanoPlanner
> > handles the optimization in dynamic programming way, not sure if the rule
> > can be applied in the right order.
> >
> > Any ideas are welcome~
> >
>
Re: rewrite an optimized RelNode
Posted by Danny Chan <yu...@gmail.com>.
Is your optimize an implementation ? Or heuristic or needs some metadata?
Yang Liu <wh...@gmail.com>于2020年3月5日 周四下午3:48写道：
> In Calcite, after optimization provided by the default VolcanoPlanner, we
> can get an optimized RelNode, but can we have a further optimization? For
> example I want to add an ElasticsearchSort or something like that to limit
> the dataset we will handle.
>
> Someone suggests we can define a RelOptRule, but since VolcanoPlanner
> handles the optimization in dynamic programming way, not sure if the rule
> can be applied in the right order.
>
> Any ideas are welcome~
>