CalcRel

[Ashutosh Chauhan <ha...@apache.org>]

Julian,

In FieldTrimmer::trimFields(setOp) there is

       newInput = CalcRel.projectMapping(newInput, remaining, null);

This returns CalcRel. While factorizing FieldTrimmer in OPTIQ-392, I left
this and didn't factorize it.

I am wondering if we need to handle this. This can potentially introduce a
CalcRel in Hive's version of Optiq plan. Harish suggested it may not be a
problem after all, because after coming out of HepPlanner, this would be
gone, so hive's ASTConverter will never encounter this. If thats true, than
nothing needs to be done and we are good. Else, following are the options:

a) Eliminate this CalcRel and have a ProjectRel there in FieldTrimmer.

b) Factorize this and introduce HiveCalcRel which Hive factory needs to
generate.

c) Keep HiveRelFieldTrimmer which overrides trimFields(setOp) and generates
HiveProjectRel there.

d) Any other options?


For me, it seems a) is preferable unless there is an inherent advantage of
having CalcRel there, else we have to do b) and need to introduce
HiveCalcRel, which I don't know if it serves any purpose other than
satisfying type system.

c) is also not very attractive since whole point of this exercise is to
throw away HiveRelFieldTrimmer and use Optiq's rule implementation.

Suggestions?


Thanks,

Ashutosh

Re: CalcRel

[Julian Hyde <ju...@hydromatic.net>]

A CalcRel is a generalization of ProjectRel that (a) uses a DAG for expressions, so that common sub-expressions are eliminated and (b) can have a filter (i.e. functionality like a FilterRel).

But I think that CalcRel.projectMapping always returns something simple enough that a ProjectRel could represent it. That is basically your option (a). Add a ProjectFactory parameter to the projectMapping method and I think we’re good.

Some historical context. CalcRel used to be the canonical form for both projects and filters during the whole planning cycle. But now CalcRel is not much used, and sub-classes of CalcRelBase such as EnumerableCalcRel are introduced later in the cycle. Common-subexpression elimination happens a bit later, which isn’t too harmful, and people can write rules targeted at project and filter patterns, which is more natural for people writing rules.

There might be one or two test failures, since intermediate forms will now involve projects rather than calcs but the ultimate plans will end up the same and I don’t think anything major will break.

Julian


On Aug 29, 2014, at 4:46 PM, Ashutosh Chauhan <ha...@apache.org> wrote:

> Julian,
> 
> In FieldTrimmer::trimFields(setOp) there is
> 
>       newInput = CalcRel.projectMapping(newInput, remaining, null);
> 
> This returns CalcRel. While factorizing FieldTrimmer in OPTIQ-392, I left
> this and didn't factorize it.
> 
> I am wondering if we need to handle this. This can potentially introduce a
> CalcRel in Hive's version of Optiq plan. Harish suggested it may not be a
> problem after all, because after coming out of HepPlanner, this would be
> gone, so hive's ASTConverter will never encounter this. If thats true, than
> nothing needs to be done and we are good. Else, following are the options:
> 
> a) Eliminate this CalcRel and have a ProjectRel there in FieldTrimmer.
> 
> b) Factorize this and introduce HiveCalcRel which Hive factory needs to
> generate.
> 
> c) Keep HiveRelFieldTrimmer which overrides trimFields(setOp) and generates
> HiveProjectRel there.
> 
> d) Any other options?
> 
> 
> For me, it seems a) is preferable unless there is an inherent advantage of
> having CalcRel there, else we have to do b) and need to introduce
> HiveCalcRel, which I don't know if it serves any purpose other than
> satisfying type system.
> 
> c) is also not very attractive since whole point of this exercise is to
> throw away HiveRelFieldTrimmer and use Optiq's rule implementation.
> 
> Suggestions?
> 
> 
> Thanks,
> 
> Ashutosh