Phoenix - Calcite integration

[Maryann Xue <ma...@gmail.com>]

Hi Julian,

After fixing the compilation error, I ran into some runtime problems with
CalciteTest, basically all caused by attempt to cast an EmptyCompositeTrait
object to RelMultipleTrait while applying rules (either ProjectRemoveRule
or PhoenixFilterRule). Guess we missed some handling of traits in our rule
implementation. Could you please help me look at the issue? I have attached
the diff file against Phoenix calcite branch.

We got our first join query to run last week (based on calcite 1.0.0), but
I have a few questions regarding the implementation:
1. Added PhoenixJoinRule, but noticed that PhoenixSortRule did something
different with the traits than other rules. What is that for? Should I also
add something special for traits in PhoenixJoinRule? And generally I would
like to know more about traits. Like what kind of traits do we usually
have? And how are we supposed to implement a trait that indicates the
ordering of an input (like for table scan, the result comes out ordered by
primary key)?
2. splitJoinCondition() only regards lhs.column1 == rhs.column2 as equi
conditions. Is there any other class/method available to recognize equi
conditions of (any LHS expression/function) == (any RHS
expression/function), for example "lhs.column1 + 3 = rhs.column2"? Or I
write my own class?
3. Also added PhoenixProjectScanMergeRule, which pushes project into a
PhoenixTableScan. Fixed an error caused by mismatch of rowtype by simply
replacing the new PhoenixTableScan's rowtype with that of the original
PhoenixProject. Does that look ok?




Thanks,
Maryann

Re: Phoenix - Calcite integration

[Julian Hyde <ju...@hydromatic.net>]

> On Mar 16, 2015, at 3:21 PM, Maryann Xue <ma...@gmail.com> wrote:
> 
> Hi Julian,
> 
> After fixing the compilation error,

I think you’re referring to the fact that you had implemented the Statistic interface then we added more methods. When implementing Statistic I suggest you use the helper methods in the Statistics class. I should have pointed that out in the javadoc for Statistic.

(Roll on Java 8 and default implementations of interface methods!)

> I ran into some runtime problems with CalciteTest, basically all caused by attempt to cast an EmptyCompositeTrait object to RelMultipleTrait while applying rules (either ProjectRemoveRule or PhoenixFilterRule). Guess we missed some handling of traits in our rule implementation. Could you please help me look at the issue? I have attached the diff file against Phoenix calcite branch.

Yes, sure. I’ll connect with you back-channel to set up a hangout.

The rules are:

1. Some traits can have multiple values: collation is the main example. Such traits implement RelMultipleTrait extends RelTrait.

2. A given RelNode has multiple instances of such traits. For example, a PhoenixFilter might be sorted on both [X] and [Y DESC, Z].

3. A RelSubset has precisely one instance of each trait. When you ask whether a RelNode r has the traits to fit into a RelSubset s, you call r.getTraitSet().satisfies(s.getTraitSet()). The satisfies method assumes that each trait on the RHS is simple. For example, the filter above satisfies [convention=PHOENIX, collation=[Y DESC]].

4. The methods boolean RelTraitSet.isSimple() and RelTraitSet.simplify() are useful but check whether you haven’t got the RelNode and RelSubset the wrong way round.

> 
> We got our first join query to run last week (based on calcite 1.0.0), but I have a few questions regarding the implementation:
> 1. Added PhoenixJoinRule, but noticed that PhoenixSortRule did something different with the traits than other rules. What is that for? Should I also add something special for traits in PhoenixJoinRule? And generally I would like to know more about traits. Like what kind of traits do we usually have? And how are we supposed to implement a trait that indicates the ordering of an input (like for table scan, the result comes out ordered by primary key)?

Give your RelNodes “create” methods, and use the “create” methods when creating other types of RelNode. In the “create” method, deduce the traits of the nodes before you create them, construct a trait set, then call the constructor. LogicalFilter.create is a good example to follow.


> 2. splitJoinCondition() only regards lhs.column1 == rhs.column2 as equi conditions. Is there any other class/method available to recognize equi conditions of (any LHS expression/function) == (any RHS expression/function), for example "lhs.column1 + 3 = rhs.column2"? Or I write my own class?

Do you know about JoinInfo.of(RelNode left, RelNode right, RexNode condition)?

> 3. Also added PhoenixProjectScanMergeRule, which pushes project into a PhoenixTableScan. Fixed an error caused by mismatch of rowtype by simply replacing the new PhoenixTableScan's rowtype with that of the original PhoenixProject. Does that look ok?

Haven’t reviewed the code but it sounds like the right thing to do.

Julian