You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Lekshmi <le...@gmail.com> on 2019/08/08 14:56:29 UTC

Filterable table

Hi,
  Can anyone explain to me what is the advantage of Filterable Table? How
can we efficiently make use of that in a Federated system?


Thanks and Regards

Lekshmi B.G
Email: lekshmibg09@gmail.com

Re: Filterable table

Posted by Danny Chan <yu...@gmail.com>.
Thanks, Albert ~

The ProjectableFilterableTable is indeed a little redundant, because we already have ProjectableTable and FilterableTable, I think we can do some refactoring to reduce such mixed in interfaces.

BTW, what do you mean by aggregate push down ? I can not figure out in which case you need such feature, can you share to us your use cases ?

Best,
Danny Chan
在 2019年8月12日 +0800 PM12:20,Albert <zi...@gmail.com>,写道:
> talking about this class reminds me of my own usage of
> ProjectableFilterable. it works.
> but this feels like a very bad design, what if I want to push down
> Aggregation ? AggregatableProjectableFilterable?
>
> I don't know the intention of the initial design, but this just doesn't
> feel right.
>
> On Mon, Aug 12, 2019 at 9:35 AM Danny Chan <yu...@gmail.com> wrote:
>
> > There is a ProjectableFilterableTable[1] that can be both used for filter
> > and projects push down, have a try :)
> >
> > [1]
> > https://github.com/apache/calcite/blob/996cdd0249a4e5086fadc27a15ccc23eb9defd8e/core/src/main/java/org/apache/calcite/schema/ProjectableFilterableTable.java#L36
> >
> > Best,
> > Danny Chan
> > 在 2019年8月12日 +0800 AM7:55,Lekshmi <le...@gmail.com>,写道:
> > > Hi Danny Chan,
> > > Thank you so much for responding to my email. Can we use, Translatable
> > > table instead of Filterable table? which is more efficient? Usecase that
> > if
> > > need to write a Project push down (Push down Project fields into table
> > scan
> > > then do Filtering) and Filter push down?
> > > Or Filterable table is enough to use Project push down as well?
> > > Thanks and Regards
> > >
> > > Lekshmi B.G
> > > Email: lekshmibg09@gmail.com
> > >
> > >
> > >
> > >
> > > On Fri, Aug 9, 2019 at 11:15 AM Danny Chan <yu...@gmail.com> wrote:
> > >
> > > > Hi, Lekshmi ~
> > > >
> > > >
> > > > The FilterableTable can be scanned with a list of filter expression,
> > that
> > > > means the filter conditions above the project + scan can be
> > > > pushed directly into the table scan[1][2], which is more efficient.
> > Here is
> > > > a CsvFilterableTable example [3].
> > > >
> > > > [1]
> > > >
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L130
> > > > [2]
> > > >
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java#L128
> > > > [3]
> > > >
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvFilterableTable.java#L65
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2019年8月8日 +0800 PM10:56,dev@calcite.apache.org,写道:
> > > > >
> > > > > Filterable
> > > >
> >
>
>
> --
> ~~~~~~~~~~~~~~~
> no mistakes
> ~~~~~~~~~~~~~~~~~~

Re: Filterable table

Posted by Michael Mior <mm...@apache.org>.
These subclasses exist to avoid creating intermediate relational
algebra expressions. You don't *need* to use any of them to push down
operations. (I didn't when creating the Cassandra adapter.) You can
create instances of different relational algebra operators for your
particular calling convention and then create rules to convert
expressions to those of your calling convention. If you want to
convert an aggregate operation, you could consider this approach.

--
Michael Mior
mmior@apache.org

Le lun. 12 août 2019 à 00:20, Albert <zi...@gmail.com> a écrit :
>
> talking about this class reminds me of my own usage of
> ProjectableFilterable. it works.
> but this feels like a very bad design, what if I want to push down
> Aggregation ? AggregatableProjectableFilterable?
>
> I don't know the intention of the initial design, but this just doesn't
> feel right.
>
> On Mon, Aug 12, 2019 at 9:35 AM Danny Chan <yu...@gmail.com> wrote:
>
> > There is a ProjectableFilterableTable[1] that can be both used for filter
> > and projects push down, have a try :)
> >
> > [1]
> > https://github.com/apache/calcite/blob/996cdd0249a4e5086fadc27a15ccc23eb9defd8e/core/src/main/java/org/apache/calcite/schema/ProjectableFilterableTable.java#L36
> >
> > Best,
> > Danny Chan
> > 在 2019年8月12日 +0800 AM7:55,Lekshmi <le...@gmail.com>,写道:
> > > Hi Danny Chan,
> > > Thank you so much for responding to my email. Can we use, Translatable
> > > table instead of Filterable table? which is more efficient? Usecase that
> > if
> > > need to write a Project push down (Push down Project fields into table
> > scan
> > > then do Filtering) and Filter push down?
> > > Or Filterable table is enough to use Project push down as well?
> > > Thanks and Regards
> > >
> > > Lekshmi B.G
> > > Email: lekshmibg09@gmail.com
> > >
> > >
> > >
> > >
> > > On Fri, Aug 9, 2019 at 11:15 AM Danny Chan <yu...@gmail.com> wrote:
> > >
> > > > Hi, Lekshmi ~
> > > >
> > > >
> > > > The FilterableTable can be scanned with a list of filter expression,
> > that
> > > > means the filter conditions above the project + scan can be
> > > > pushed directly into the table scan[1][2], which is more efficient.
> > Here is
> > > > a CsvFilterableTable example [3].
> > > >
> > > > [1]
> > > >
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L130
> > > > [2]
> > > >
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java#L128
> > > > [3]
> > > >
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvFilterableTable.java#L65
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2019年8月8日 +0800 PM10:56,dev@calcite.apache.org,写道:
> > > > >
> > > > > Filterable
> > > >
> >
>
>
> --
> ~~~~~~~~~~~~~~~
> no mistakes
> ~~~~~~~~~~~~~~~~~~

Re: Filterable table

Posted by Albert <zi...@gmail.com>.
talking about this class reminds me of my own usage of
ProjectableFilterable. it works.
but this feels like a very bad design, what if I want to push down
Aggregation ? AggregatableProjectableFilterable?

I don't know the intention of the initial design, but this just doesn't
feel right.

On Mon, Aug 12, 2019 at 9:35 AM Danny Chan <yu...@gmail.com> wrote:

> There is a ProjectableFilterableTable[1] that can be both used for filter
> and projects push down, have a try :)
>
> [1]
> https://github.com/apache/calcite/blob/996cdd0249a4e5086fadc27a15ccc23eb9defd8e/core/src/main/java/org/apache/calcite/schema/ProjectableFilterableTable.java#L36
>
> Best,
> Danny Chan
> 在 2019年8月12日 +0800 AM7:55,Lekshmi <le...@gmail.com>,写道:
> > Hi Danny Chan,
> > Thank you so much for responding to my email. Can we use, Translatable
> > table instead of Filterable table? which is more efficient? Usecase that
> if
> > need to write a Project push down (Push down Project fields into table
> scan
> > then do Filtering) and Filter push down?
> > Or Filterable table is enough to use Project push down as well?
> > Thanks and Regards
> >
> > Lekshmi B.G
> > Email: lekshmibg09@gmail.com
> >
> >
> >
> >
> > On Fri, Aug 9, 2019 at 11:15 AM Danny Chan <yu...@gmail.com> wrote:
> >
> > > Hi, Lekshmi ~
> > >
> > >
> > > The FilterableTable can be scanned with a list of filter expression,
> that
> > > means the filter conditions above the project + scan can be
> > > pushed directly into the table scan[1][2], which is more efficient.
> Here is
> > > a CsvFilterableTable example [3].
> > >
> > > [1]
> > >
> https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L130
> > > [2]
> > >
> https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java#L128
> > > [3]
> > >
> https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvFilterableTable.java#L65
> > >
> > > Best,
> > > Danny Chan
> > > 在 2019年8月8日 +0800 PM10:56,dev@calcite.apache.org,写道:
> > > >
> > > > Filterable
> > >
>


-- 
~~~~~~~~~~~~~~~
no mistakes
~~~~~~~~~~~~~~~~~~

Re: Filterable table

Posted by Danny Chan <yu...@gmail.com>.
There is a ProjectableFilterableTable[1] that can be both used for filter and projects push down, have a try :)

[1] https://github.com/apache/calcite/blob/996cdd0249a4e5086fadc27a15ccc23eb9defd8e/core/src/main/java/org/apache/calcite/schema/ProjectableFilterableTable.java#L36

Best,
Danny Chan
在 2019年8月12日 +0800 AM7:55,Lekshmi <le...@gmail.com>,写道:
> Hi Danny Chan,
> Thank you so much for responding to my email. Can we use, Translatable
> table instead of Filterable table? which is more efficient? Usecase that if
> need to write a Project push down (Push down Project fields into table scan
> then do Filtering) and Filter push down?
> Or Filterable table is enough to use Project push down as well?
> Thanks and Regards
>
> Lekshmi B.G
> Email: lekshmibg09@gmail.com
>
>
>
>
> On Fri, Aug 9, 2019 at 11:15 AM Danny Chan <yu...@gmail.com> wrote:
>
> > Hi, Lekshmi ~
> >
> >
> > The FilterableTable can be scanned with a list of filter expression, that
> > means the filter conditions above the project + scan can be
> > pushed directly into the table scan[1][2], which is more efficient. Here is
> > a CsvFilterableTable example [3].
> >
> > [1]
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L130
> > [2]
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java#L128
> > [3]
> > https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvFilterableTable.java#L65
> >
> > Best,
> > Danny Chan
> > 在 2019年8月8日 +0800 PM10:56,dev@calcite.apache.org,写道:
> > >
> > > Filterable
> >

Re: Filterable table

Posted by Lekshmi <le...@gmail.com>.
Hi Danny Chan,
 Thank you so much for responding to my email.  Can we use, Translatable
table instead of Filterable table? which is more efficient? Usecase that if
need to write a Project push down (Push down Project fields into table scan
then do Filtering) and Filter push down?
Or Filterable table is enough to use Project push down as well?
Thanks and Regards

Lekshmi B.G
Email: lekshmibg09@gmail.com




On Fri, Aug 9, 2019 at 11:15 AM Danny Chan <yu...@gmail.com> wrote:

> Hi, Lekshmi ~
>
>
> The FilterableTable can be scanned with a list of filter expression, that
> means the filter conditions above the project + scan can be
> pushed directly into the table scan[1][2], which is more efficient. Here is
> a CsvFilterableTable example [3].
>
> [1]
> https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L130
> [2]
> https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java#L128
> [3]
> https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvFilterableTable.java#L65
>
> Best,
> Danny Chan
> 在 2019年8月8日 +0800 PM10:56,dev@calcite.apache.org,写道:
> >
> > Filterable
>

Re: Filterable table

Posted by Danny Chan <yu...@gmail.com>.
Hi, Lekshmi ~


The FilterableTable can be scanned with a list of filter expression, that means the filter conditions above the project + scan can be pushed directly into the table scan[1][2], which is more efficient. Here is a CsvFilterableTable example [3].

[1] https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L130
[2] https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java#L128
[3] https://github.com/apache/calcite/blob/602b2a10e3f81888c0759ca69f7415497124ef82/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvFilterableTable.java#L65

Best,
Danny Chan
在 2019年8月8日 +0800 PM10:56,dev@calcite.apache.org,写道:
>
> Filterable