You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by JiaTao Tao <ta...@gmail.com> on 2020/10/21 06:48:49 UTC
RelBuilder#project should try keep alias in optimizer
Hi fellows
I found in some place, we just call `project(Iterable<? extends RexNode>
nodes) `, but this will lose alias, why don't we try to call
`project(Iterable<?
extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
Regards!
Aron Tao
Re: RelBuilder#project should try keep alias in optimizer
Posted by Julian Hyde <jh...@apache.org>.
There are two distinct purposes for aliases in RelBuilder:
1. Allow you to build expressions referencing columns and relations
that you just created in the same RelBuilder.
2. Column aliases become field names in the row type of the relational
expressions.
#2 is best-effort. For example, if you request a Project with fields
(x, y) and your Project turns out after simplification to be the
identity, then RelBuilder will not create a new Project; it will
re-use the old one, which may have different column names.
#1 is reliable. If you have just created a project with columns x and
y, then you can safely call field("x") to reference field 0.
RelBuilder maintains an internal map of field names and relation
aliases in order to make that work, even if the relational expressions
have different field names.
If you want to know more about this, read the test cases in
RelBuilderTest carefully.
Julian
On Thu, Oct 22, 2020 at 2:49 AM JiaTao Tao <ta...@gmail.com> wrote:
>
> What do you mean?
> What I mean is to use "project(rexNodes, filedNames)", we can keep the
> alias, take https://issues.apache.org/jira/browse/CALCITE-4037, we just
> replace the "project(rexNodes)" to "project(rexNodes, filedNames)", then we
> can keep the alias. But as said "keep the alias, there are many cases that
> can not be promoted.", so do this mean we should use "project(rexNodes)",
> but there are many rule use "project(rexNodes, filedNames)", does this meet
> the expectations?
>
> Regards!
>
> Aron Tao
>
>
> Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午4:40写道：
>
> > Did you read the javadoc for the two ‘project’ methods?
> >
> > Julian
> >
> >
> > > On Oct 22, 2020, at 1:15 AM, JiaTao Tao <ta...@gmail.com> wrote:
> > >
> > > Some rules we use `RelBuilder project(Iterable<? extends RexNode> nodes)
> > `,
> > > sometimes use `RelBuilder project(Iterable<? extends RexNode> nodes,
> > > Iterable<String>
> > > fieldNames)`, and this really makes me confused.
> > >
> > >
> > > Regards!
> > >
> > > Aron Tao
> > >
> > >
> > > JiaTao Tao <ta...@gmail.com> 于2020年10月22日周四 下午4:13写道：
> > >
> > >> Thanks, Julian
> > >> One more question is that I can see many
> > >> RelBuilder#project(java.lang.Iterable<? extends
> > >> org.apache.calcite.rex.RexNode>, java.lang.Iterable<java.lang.String>)
> > >> calling in some rules, does this meets the expect, if we don't want to
> > keep
> > >> alias in optimizer.
> > >>
> > >>
> > >> Regards!
> > >>
> > >> Aron Tao
> > >>
> > >>
> > >> Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午3:29写道：
> > >>
> > >>> See my comments in 1584. I think that answers your question.
> > >>>
> > >>> Julian
> > >>>
> > >>>> On Oct 21, 2020, at 23:35, JiaTao Tao <ta...@gmail.com> wrote:
> > >>>>
> > >>>> Hi Danny
> > >>>> Thanks for your reply, can you explain "Calcite only optimize based on
> > >>> the
> > >>>> field input refs, if we always keep the alias, there are many cases
> > that
> > >>>> can not be promoted" a bit? In my mind, alias just field names,
> > doesn't
> > >>>> affect the input refs?
> > >>>>
> > >>>> Regards!
> > >>>>
> > >>>> Aron Tao
> > >>>>
> > >>>>
> > >>>> Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
> > >>>>
> > >>>>> During planning, Calcite only optimize based on the field input refs,
> > >>> if
> > >>>>> we always keep the alias, there are many cases that can not be
> > >>> promoted. I
> > >>>>> collected some cases that already tracked by the JIRA issue:
> > >>>>>
> > >>>>> [1] https://issues.apache.org/jira/browse/CALCITE-4037
> > >>>>> [2] https://issues.apache.org/jira/browse/CALCITE-3662
> > >>>>> [3] https://issues.apache.org/jira/browse/CALCITE-1584
> > >>>>>
> > >>>>> Best,
> > >>>>> Danny Chan
> > >>>>> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
> > >>>>>> Hi fellows
> > >>>>>> I found in some place, we just call `project(Iterable<? extends
> > >>> RexNode>
> > >>>>>> nodes) `, but this will lose alias, why don't we try to call
> > >>>>>> `project(Iterable<?
> > >>>>>> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep
> > alias?
> > >>>>>>
> > >>>>>> Regards!
> > >>>>>>
> > >>>>>> Aron Tao
> > >>>>>
> > >>>
> > >>
> >
> >
Re: RelBuilder#project should try keep alias in optimizer
Posted by JiaTao Tao <ta...@gmail.com>.
What do you mean?
What I mean is to use "project(rexNodes, filedNames)", we can keep the
alias, take https://issues.apache.org/jira/browse/CALCITE-4037, we just
replace the "project(rexNodes)" to "project(rexNodes, filedNames)", then we
can keep the alias. But as said "keep the alias, there are many cases that
can not be promoted.", so do this mean we should use "project(rexNodes)",
but there are many rule use "project(rexNodes, filedNames)", does this meet
the expectations?
Regards!
Aron Tao
Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午4:40写道：
> Did you read the javadoc for the two ‘project’ methods?
>
> Julian
>
>
> > On Oct 22, 2020, at 1:15 AM, JiaTao Tao <ta...@gmail.com> wrote:
> >
> > Some rules we use `RelBuilder project(Iterable<? extends RexNode> nodes)
> `,
> > sometimes use `RelBuilder project(Iterable<? extends RexNode> nodes,
> > Iterable<String>
> > fieldNames)`, and this really makes me confused.
> >
> >
> > Regards!
> >
> > Aron Tao
> >
> >
> > JiaTao Tao <ta...@gmail.com> 于2020年10月22日周四 下午4:13写道：
> >
> >> Thanks, Julian
> >> One more question is that I can see many
> >> RelBuilder#project(java.lang.Iterable<? extends
> >> org.apache.calcite.rex.RexNode>, java.lang.Iterable<java.lang.String>)
> >> calling in some rules, does this meets the expect, if we don't want to
> keep
> >> alias in optimizer.
> >>
> >>
> >> Regards!
> >>
> >> Aron Tao
> >>
> >>
> >> Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午3:29写道：
> >>
> >>> See my comments in 1584. I think that answers your question.
> >>>
> >>> Julian
> >>>
> >>>> On Oct 21, 2020, at 23:35, JiaTao Tao <ta...@gmail.com> wrote:
> >>>>
> >>>> Hi Danny
> >>>> Thanks for your reply, can you explain "Calcite only optimize based on
> >>> the
> >>>> field input refs, if we always keep the alias, there are many cases
> that
> >>>> can not be promoted" a bit? In my mind, alias just field names,
> doesn't
> >>>> affect the input refs?
> >>>>
> >>>> Regards!
> >>>>
> >>>> Aron Tao
> >>>>
> >>>>
> >>>> Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
> >>>>
> >>>>> During planning, Calcite only optimize based on the field input refs,
> >>> if
> >>>>> we always keep the alias, there are many cases that can not be
> >>> promoted. I
> >>>>> collected some cases that already tracked by the JIRA issue:
> >>>>>
> >>>>> [1] https://issues.apache.org/jira/browse/CALCITE-4037
> >>>>> [2] https://issues.apache.org/jira/browse/CALCITE-3662
> >>>>> [3] https://issues.apache.org/jira/browse/CALCITE-1584
> >>>>>
> >>>>> Best,
> >>>>> Danny Chan
> >>>>> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
> >>>>>> Hi fellows
> >>>>>> I found in some place, we just call `project(Iterable<? extends
> >>> RexNode>
> >>>>>> nodes) `, but this will lose alias, why don't we try to call
> >>>>>> `project(Iterable<?
> >>>>>> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep
> alias?
> >>>>>>
> >>>>>> Regards!
> >>>>>>
> >>>>>> Aron Tao
> >>>>>
> >>>
> >>
>
>
Re: RelBuilder#project should try keep alias in optimizer
Posted by Julian Hyde <jh...@gmail.com>.
Did you read the javadoc for the two ‘project’ methods?
Julian
> On Oct 22, 2020, at 1:15 AM, JiaTao Tao <ta...@gmail.com> wrote:
>
> Some rules we use `RelBuilder project(Iterable<? extends RexNode> nodes) `,
> sometimes use `RelBuilder project(Iterable<? extends RexNode> nodes,
> Iterable<String>
> fieldNames)`, and this really makes me confused.
>
>
> Regards!
>
> Aron Tao
>
>
> JiaTao Tao <ta...@gmail.com> 于2020年10月22日周四 下午4:13写道：
>
>> Thanks, Julian
>> One more question is that I can see many
>> RelBuilder#project(java.lang.Iterable<? extends
>> org.apache.calcite.rex.RexNode>, java.lang.Iterable<java.lang.String>)
>> calling in some rules, does this meets the expect, if we don't want to keep
>> alias in optimizer.
>>
>>
>> Regards!
>>
>> Aron Tao
>>
>>
>> Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午3:29写道：
>>
>>> See my comments in 1584. I think that answers your question.
>>>
>>> Julian
>>>
>>>> On Oct 21, 2020, at 23:35, JiaTao Tao <ta...@gmail.com> wrote:
>>>>
>>>> Hi Danny
>>>> Thanks for your reply, can you explain "Calcite only optimize based on
>>> the
>>>> field input refs, if we always keep the alias, there are many cases that
>>>> can not be promoted" a bit? In my mind, alias just field names, doesn't
>>>> affect the input refs?
>>>>
>>>> Regards!
>>>>
>>>> Aron Tao
>>>>
>>>>
>>>> Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
>>>>
>>>>> During planning, Calcite only optimize based on the field input refs,
>>> if
>>>>> we always keep the alias, there are many cases that can not be
>>> promoted. I
>>>>> collected some cases that already tracked by the JIRA issue:
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/CALCITE-4037
>>>>> [2] https://issues.apache.org/jira/browse/CALCITE-3662
>>>>> [3] https://issues.apache.org/jira/browse/CALCITE-1584
>>>>>
>>>>> Best,
>>>>> Danny Chan
>>>>> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
>>>>>> Hi fellows
>>>>>> I found in some place, we just call `project(Iterable<? extends
>>> RexNode>
>>>>>> nodes) `, but this will lose alias, why don't we try to call
>>>>>> `project(Iterable<?
>>>>>> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
>>>>>>
>>>>>> Regards!
>>>>>>
>>>>>> Aron Tao
>>>>>
>>>
>>
Re: RelBuilder#project should try keep alias in optimizer
Posted by JiaTao Tao <ta...@gmail.com>.
Some rules we use `RelBuilder project(Iterable<? extends RexNode> nodes) `,
sometimes use `RelBuilder project(Iterable<? extends RexNode> nodes,
Iterable<String>
fieldNames)`, and this really makes me confused.
Regards!
Aron Tao
JiaTao Tao <ta...@gmail.com> 于2020年10月22日周四 下午4:13写道：
> Thanks, Julian
> One more question is that I can see many
> RelBuilder#project(java.lang.Iterable<? extends
> org.apache.calcite.rex.RexNode>, java.lang.Iterable<java.lang.String>)
> calling in some rules, does this meets the expect, if we don't want to keep
> alias in optimizer.
>
>
> Regards!
>
> Aron Tao
>
>
> Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午3:29写道：
>
>> See my comments in 1584. I think that answers your question.
>>
>> Julian
>>
>> > On Oct 21, 2020, at 23:35, JiaTao Tao <ta...@gmail.com> wrote:
>> >
>> > Hi Danny
>> > Thanks for your reply, can you explain "Calcite only optimize based on
>> the
>> > field input refs, if we always keep the alias, there are many cases that
>> > can not be promoted" a bit? In my mind, alias just field names, doesn't
>> > affect the input refs?
>> >
>> > Regards!
>> >
>> > Aron Tao
>> >
>> >
>> > Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
>> >
>> >> During planning, Calcite only optimize based on the field input refs,
>> if
>> >> we always keep the alias, there are many cases that can not be
>> promoted. I
>> >> collected some cases that already tracked by the JIRA issue:
>> >>
>> >> [1] https://issues.apache.org/jira/browse/CALCITE-4037
>> >> [2] https://issues.apache.org/jira/browse/CALCITE-3662
>> >> [3] https://issues.apache.org/jira/browse/CALCITE-1584
>> >>
>> >> Best,
>> >> Danny Chan
>> >> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
>> >>> Hi fellows
>> >>> I found in some place, we just call `project(Iterable<? extends
>> RexNode>
>> >>> nodes) `, but this will lose alias, why don't we try to call
>> >>> `project(Iterable<?
>> >>> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
>> >>>
>> >>> Regards!
>> >>>
>> >>> Aron Tao
>> >>
>>
>
Re: RelBuilder#project should try keep alias in optimizer
Posted by JiaTao Tao <ta...@gmail.com>.
Thanks, Julian
One more question is that I can see many
RelBuilder#project(java.lang.Iterable<? extends
org.apache.calcite.rex.RexNode>, java.lang.Iterable<java.lang.String>)
calling in some rules, does this meets the expect, if we don't want to keep
alias in optimizer.
Regards!
Aron Tao
Julian Hyde <jh...@gmail.com> 于2020年10月22日周四 下午3:29写道：
> See my comments in 1584. I think that answers your question.
>
> Julian
>
> > On Oct 21, 2020, at 23:35, JiaTao Tao <ta...@gmail.com> wrote:
> >
> > Hi Danny
> > Thanks for your reply, can you explain "Calcite only optimize based on
> the
> > field input refs, if we always keep the alias, there are many cases that
> > can not be promoted" a bit? In my mind, alias just field names, doesn't
> > affect the input refs?
> >
> > Regards!
> >
> > Aron Tao
> >
> >
> > Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
> >
> >> During planning, Calcite only optimize based on the field input refs, if
> >> we always keep the alias, there are many cases that can not be
> promoted. I
> >> collected some cases that already tracked by the JIRA issue:
> >>
> >> [1] https://issues.apache.org/jira/browse/CALCITE-4037
> >> [2] https://issues.apache.org/jira/browse/CALCITE-3662
> >> [3] https://issues.apache.org/jira/browse/CALCITE-1584
> >>
> >> Best,
> >> Danny Chan
> >> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
> >>> Hi fellows
> >>> I found in some place, we just call `project(Iterable<? extends
> RexNode>
> >>> nodes) `, but this will lose alias, why don't we try to call
> >>> `project(Iterable<?
> >>> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
> >>>
> >>> Regards!
> >>>
> >>> Aron Tao
> >>
>
Re: RelBuilder#project should try keep alias in optimizer
Posted by Julian Hyde <jh...@gmail.com>.
See my comments in 1584. I think that answers your question.
Julian
> On Oct 21, 2020, at 23:35, JiaTao Tao <ta...@gmail.com> wrote:
>
> Hi Danny
> Thanks for your reply, can you explain "Calcite only optimize based on the
> field input refs, if we always keep the alias, there are many cases that
> can not be promoted" a bit? In my mind, alias just field names, doesn't
> affect the input refs?
>
> Regards!
>
> Aron Tao
>
>
> Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
>
>> During planning, Calcite only optimize based on the field input refs, if
>> we always keep the alias, there are many cases that can not be promoted. I
>> collected some cases that already tracked by the JIRA issue:
>>
>> [1] https://issues.apache.org/jira/browse/CALCITE-4037
>> [2] https://issues.apache.org/jira/browse/CALCITE-3662
>> [3] https://issues.apache.org/jira/browse/CALCITE-1584
>>
>> Best,
>> Danny Chan
>> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
>>> Hi fellows
>>> I found in some place, we just call `project(Iterable<? extends RexNode>
>>> nodes) `, but this will lose alias, why don't we try to call
>>> `project(Iterable<?
>>> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
>>>
>>> Regards!
>>>
>>> Aron Tao
>>
Re: RelBuilder#project should try keep alias in optimizer
Posted by JiaTao Tao <ta...@gmail.com>.
Hi Danny
Thanks for your reply, can you explain "Calcite only optimize based on the
field input refs, if we always keep the alias, there are many cases that
can not be promoted" a bit? In my mind, alias just field names, doesn't
affect the input refs?
Regards!
Aron Tao
Danny Chan <yu...@gmail.com> 于2020年10月22日周四 上午11:29写道：
> During planning, Calcite only optimize based on the field input refs, if
> we always keep the alias, there are many cases that can not be promoted. I
> collected some cases that already tracked by the JIRA issue:
>
> [1] https://issues.apache.org/jira/browse/CALCITE-4037
> [2] https://issues.apache.org/jira/browse/CALCITE-3662
> [3] https://issues.apache.org/jira/browse/CALCITE-1584
>
> Best,
> Danny Chan
> 在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
> > Hi fellows
> > I found in some place, we just call `project(Iterable<? extends RexNode>
> > nodes) `, but this will lose alias, why don't we try to call
> > `project(Iterable<?
> > extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
> >
> > Regards!
> >
> > Aron Tao
>
Re: RelBuilder#project should try keep alias in optimizer
Posted by Danny Chan <yu...@gmail.com>.
During planning, Calcite only optimize based on the field input refs, if we always keep the alias, there are many cases that can not be promoted. I collected some cases that already tracked by the JIRA issue:
[1] https://issues.apache.org/jira/browse/CALCITE-4037
[2] https://issues.apache.org/jira/browse/CALCITE-3662
[3] https://issues.apache.org/jira/browse/CALCITE-1584
Best,
Danny Chan
在 2020年10月21日 +0800 PM2:49，JiaTao Tao <ta...@gmail.com>，写道：
> Hi fellows
> I found in some place, we just call `project(Iterable<? extends RexNode>
> nodes) `, but this will lose alias, why don't we try to call
> `project(Iterable<?
> extends RexNode> nodes, Iterable<String> fieldNames) ` to keep alias?
>
> Regards!
>
> Aron Tao