You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by MinJi Kim <mi...@alum.mit.edu> on 2016/03/10 07:34:05 UTC
question about RelDistribution conversion
Hello Calcite-experts --
I am new to Calcite (and optimzers in general). I am trying to
understand how other RelTrait conversions work in Calcite. It looks
like Conventions are treated specially and handled at the root, but I am
curious to understand how other traits (Distribution/Collation) are
handled. I wrote a simple test case to help me understand and to see if
I could convert a simple two node tree with convention.NONE to
convention.PHYS with some RelDistribution manipulations
(TraitConversionTest.java). But the test fails with "could not be
implemented". The following transformations were what I envisioned, but
it looks like the ConvertRelDistributionTraitDef.convert() is not
effective. Could somebody give me pointers as to what should be done to
see the following transformations?
TraitConversionTest.java
https://github.com/minji-kim/calcite/commit/d51d93e2bdc3febab0410d1c0f2c61b6cb8af46f
Thanks!
Minji
1) original:
NoneSingleRel
|
NoneLeafRel
2) desired result:
RandomSingleRel (PHYS.random)
|
BridgeRel (PHYS.random)
|
SingletonLeafRel (PHYS.single)
=====
org.apache.calcite.plan.RelOptPlanner$CannotPlanException: Node
[rel#5:Subset#1.PHYS.any] could not be implemented; planner state:
Root: rel#5:Subset#1.PHYS.any
Original rel:
Sets:
Set#0, type: RecordType(JavaType(void) this)
rel#2:Subset#0.NONE.any, best=null, importance=0.81
rel#0:NoneLeafRel.NONE.any(label=a), rowcount=1.0, cumulative
cost={inf}
rel#7:Subset#0.PHYS.random, best=null, importance=0.81
rel#12:Subset#0.PHYS.single, best=rel#11, importance=0.405
rel#11:SingletonLeafRel.PHYS.single(label=a), rowcount=1.0,
cumulative cost={tiny}
Set#1, type: RecordType(JavaType(void) this)
rel#4:Subset#1.NONE.any, best=null, importance=0.9
rel#3:NoneSingleRel.NONE.any(input=rel#2:Subset#0.NONE.any),
rowcount=1.0, cumulative cost={inf}
rel#5:Subset#1.PHYS.any, best=null, importance=1.0
rel#6:AbstractConverter.PHYS.any(input=rel#4:Subset#1.NONE.any,convention=PHYS,dist=any),
rowcount=1.0, cumulative cost={inf}
rel#10:AbstractConverter.PHYS.any(input=rel#9:Subset#1.PHYS.random,convention=PHYS,dist=any),
rowcount=1.0, cumulative cost={inf}
rel#8:RandomSingleRel.PHYS.random(input=rel#7:Subset#0.PHYS.random),
rowcount=1.0, cumulative cost={inf}
rel#9:Subset#1.PHYS.random, best=null, importance=0.9
rel#8:RandomSingleRel.PHYS.random(input=rel#7:Subset#0.PHYS.random),
rowcount=1.0, cumulative cost={inf}
question about Distribution and Collation RelTrait conversion
Posted by MinJi Kim <mi...@alum.mit.edu>.
Hello calcite-experts --
It looks like RelDistribution and RelCollation conversions are not
happening (perhaps it is any RelTrait other than Convention) in
VolcanoPlanner. Could somebody more familiar with Calcite to confirm
this? Or, give me pointers as to why these tests are failing? I would
like to get these RelTrait conversions to work with the RelTraitDef.
I wrote two tests, and they both fail without the following changes in
RelSet.java (which I have commented out in the commit below).
* CollationConversionTest (for RelCollation)
* TraitConversionTest (for RelDistribution)
https://github.com/minji-kim/calcite/commit/2d672ff6bc0f97b27bee6bedf18e4c5d8b83541b
This change is related to changesets 3b55c35, b312031.
I made some changes to RelSet to see if I could get the
collation/distribution conversions to work with VolcanoPlanner. It
looks like the original code of adding AbstractConverter was a bit too
aggressive,added too many AbstractConverters, and increased the search
space. There is a nice method in *RelTraitDef.canConvert()*, which
isn't used anywhere, that can help prune out some AbstractConverters
that are not needed. Also, from my understanding, it looks like
RelTraitDef only converts only one RelTraitDef, so I think before adding
the AbstractConverters, it would be good to check that the difference in
the RelTraitSet is of size 1 and the RelTraitDef.canConvert() return
true. I am not sure if this will have much of an impact in the
performance concerns (i.e. significantly reduce the number of
AbstractConverters as pointed out in 3b55c35), but thought it was worth
a try.
Thanks!
Minji
On 3/9/16 10:34 PM, MinJi Kim wrote:
> Hello Calcite-experts --
>
> I am new to Calcite (and optimzers in general). I am trying to
> understand how other RelTrait conversions work in Calcite. It looks
> like Conventions are treated specially and handled at the root, but I
> am curious to understand how other traits (Distribution/Collation) are
> handled. I wrote a simple test case to help me understand and to see
> if I could convert a simple two node tree with convention.NONE to
> convention.PHYS with some RelDistribution manipulations
> (TraitConversionTest.java). But the test fails with "could not be
> implemented". The following transformations were what I envisioned,
> but it looks like the ConvertRelDistributionTraitDef.convert() is not
> effective. Could somebody give me pointers as to what should be done
> to see the following transformations?
>
> TraitConversionTest.java
> https://github.com/minji-kim/calcite/commit/d51d93e2bdc3febab0410d1c0f2c61b6cb8af46f
>
> Thanks!
> Minji
>
> 1) original:
> NoneSingleRel
> |
> NoneLeafRel
>
> 2) desired result:
> RandomSingleRel (PHYS.random)
> |
> BridgeRel (PHYS.random)
> |
> SingletonLeafRel (PHYS.single)
>
>
> =====
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: Node
> [rel#5:Subset#1.PHYS.any] could not be implemented; planner state:
>
> Root: rel#5:Subset#1.PHYS.any
> Original rel:
>
> Sets:
> Set#0, type: RecordType(JavaType(void) this)
> rel#2:Subset#0.NONE.any, best=null, importance=0.81
> rel#0:NoneLeafRel.NONE.any(label=a), rowcount=1.0, cumulative
> cost={inf}
> rel#7:Subset#0.PHYS.random, best=null, importance=0.81
> rel#12:Subset#0.PHYS.single, best=rel#11, importance=0.405
> rel#11:SingletonLeafRel.PHYS.single(label=a), rowcount=1.0,
> cumulative cost={tiny}
> Set#1, type: RecordType(JavaType(void) this)
> rel#4:Subset#1.NONE.any, best=null, importance=0.9
> rel#3:NoneSingleRel.NONE.any(input=rel#2:Subset#0.NONE.any),
> rowcount=1.0, cumulative cost={inf}
> rel#5:Subset#1.PHYS.any, best=null, importance=1.0
> rel#6:AbstractConverter.PHYS.any(input=rel#4:Subset#1.NONE.any,convention=PHYS,dist=any),
> rowcount=1.0, cumulative cost={inf}
> rel#10:AbstractConverter.PHYS.any(input=rel#9:Subset#1.PHYS.random,convention=PHYS,dist=any),
> rowcount=1.0, cumulative cost={inf}
> rel#8:RandomSingleRel.PHYS.random(input=rel#7:Subset#0.PHYS.random),
> rowcount=1.0, cumulative cost={inf}
> rel#9:Subset#1.PHYS.random, best=null, importance=0.9
> rel#8:RandomSingleRel.PHYS.random(input=rel#7:Subset#0.PHYS.random),
> rowcount=1.0, cumulative cost={inf}