questions on volcano planner

[吴晓菊 <ch...@gmail.com>]

Hi Julian,

I'm a developer from eBay(the same org with Kylin), we are now looking into
your implementation of Volcano planner and trying to implement a new
optimizer in Spark. We meet some questions when looking into your code, so
I wrote this email and hope you can help us understand your design:

1. When applying Commute and Associate rules to get alternative reorder
plans, we need to handle join conditions, for example,
      ((A join B on a=b) join C on b=c),
   it can be transformed to
      ((A join C on a=c) join B on b=c)
   if we didn't split the 2 sides of EqualTo, it could be:
      ((A join C) join B on a=b and b=c)    then A join C becomes a
Cartesian Join
   By looking into your code, seems you didn't split the 2 sides of
EqualTo, how did you avoid the Cartesian Join?

2. I try to debug into the JoinAssociateRule but failed even though I run
all your test cases. That's my second question.

Thanks a lot for your time in advance!

Xiaoju Wu
Phone：+86 17717640807

Re: questions on volcano planner

[吴晓菊 <ch...@gmail.com>]

I found there's a rule named JoinPushThroughJoinRule, which looks like
JoinAssociateRule, while JoinAssociateRule is disabled by default.
Also there's a ticket https://issues.apache.org/jira/browse/CALCITE-346

So what's the purpose of JoinPushThroughJoinRule and why JoinAssociateRule
is disabled by default?
According to the above ticket, does it mean, JoinPushThroughJoinRule will
not generate all possible plans but JoinAssociateRule will? So we use
JoinPushThroughJoinRule by default to not make the search space too large?

Xiaoju Wu
Phone：+86 17717640807



Renjie Liu <li...@gmail.com> 于2018年9月18日周二 下午3:45写道：

> Hi:
> 1. The Cartesian join maybe discarded by the planner since its cost is much
> higher than other?
>
> On Tue, Sep 18, 2018 at 1:49 PM 吴晓菊 <ch...@gmail.com> wrote:
>
> > Hi Julian,
> >
> > I'm a developer from eBay(the same org with Kylin), we are now looking
> into
> > your implementation of Volcano planner and trying to implement a new
> > optimizer in Spark. We meet some questions when looking into your code,
> so
> > I wrote this email and hope you can help us understand your design:
> >
> > 1. When applying Commute and Associate rules to get alternative reorder
> > plans, we need to handle join conditions, for example,
> >       ((A join B on a=b) join C on b=c),
> >    it can be transformed to
> >       ((A join C on a=c) join B on b=c)
> >    if we didn't split the 2 sides of EqualTo, it could be:
> >       ((A join C) join B on a=b and b=c)    then A join C becomes a
> > Cartesian Join
> >    By looking into your code, seems you didn't split the 2 sides of
> > EqualTo, how did you avoid the Cartesian Join?
> >
> > 2. I try to debug into the JoinAssociateRule but failed even though I run
> > all your test cases. That's my second question.
> >
> > Thanks a lot for your time in advance!
> >
> > Xiaoju Wu
> > Phone：+86 17717640807 <177%201764%200807>
> >
> --
> Liu, Renjie
> Software Engineer, MVAD
>

Re: questions on volcano planner

[Renjie Liu <li...@gmail.com>]

Hi:
1. The Cartesian join maybe discarded by the planner since its cost is much
higher than other?

On Tue, Sep 18, 2018 at 1:49 PM 吴晓菊 <ch...@gmail.com> wrote:

> Hi Julian,
>
> I'm a developer from eBay(the same org with Kylin), we are now looking into
> your implementation of Volcano planner and trying to implement a new
> optimizer in Spark. We meet some questions when looking into your code, so
> I wrote this email and hope you can help us understand your design:
>
> 1. When applying Commute and Associate rules to get alternative reorder
> plans, we need to handle join conditions, for example,
>       ((A join B on a=b) join C on b=c),
>    it can be transformed to
>       ((A join C on a=c) join B on b=c)
>    if we didn't split the 2 sides of EqualTo, it could be:
>       ((A join C) join B on a=b and b=c)    then A join C becomes a
> Cartesian Join
>    By looking into your code, seems you didn't split the 2 sides of
> EqualTo, how did you avoid the Cartesian Join?
>
> 2. I try to debug into the JoinAssociateRule but failed even though I run
> all your test cases. That's my second question.
>
> Thanks a lot for your time in advance!
>
> Xiaoju Wu
> Phone：+86 17717640807 <177%201764%200807>
>
-- 
Liu, Renjie
Software Engineer, MVAD