Optiq query push down question

[Abishek Baskaran <ab...@gmail.com>]

Hi,
I have a question regarding Optiq functionality during push down operation.

Let's say I have a query like:
   select name, age from users where country = "India"

I have a filter rule that captures the filter as:
   (country) = India

I push down this filter to my data source, but since I have not implemented
filters in my data source I return back all rows to Optiq including country
<> India.

I find that Optiq takes care of filtering country = India. My question is:
Looks like in spite of pushing down a filter, Optiq attempts another round
of filtration. Though this is good data quality wise, wont this cause a
performance impact as filter is being applied in 2 places (Data source and
Optiq)? Is there a way to turn off operation at Optiq layer when the
operation has been pushed down.

Regards,
Abishek

Re: Optiq query push down question

[Abishek Baskaran <ab...@gmail.com>]

I got the problem. Looks like only the project rule got fired I guess. I
see that now if Optiq pushes down an operation, It doesn't redo it.

Regards,
Abishek


On Tue, Aug 5, 2014 at 10:20 AM, Abishek Baskaran <
abishek.baskaran@gmail.com> wrote:

> Hi,
> I have a question regarding Optiq functionality during push down operation.
>
> Let's say I have a query like:
>    select name, age from users where country = "India"
>
> I have a filter rule that captures the filter as:
>    (country) = India
>
> I push down this filter to my data source, but since I have not
> implemented filters in my data source I return back all rows to Optiq
> including country <> India.
>
> I find that Optiq takes care of filtering country = India. My question is:
> Looks like in spite of pushing down a filter, Optiq attempts another round
> of filtration. Though this is good data quality wise, wont this cause a
> performance impact as filter is being applied in 2 places (Data source and
> Optiq)? Is there a way to turn off operation at Optiq layer when the
> operation has been pushed down.
>
> Regards,
> Abishek
>
>
>

Re: Optiq query push down question

[Vladimir Sitnikov <si...@gmail.com>]

Hi,

Can you please share the testcase and/or the plan reported by optiq?

The plan can be captured either in VolcanoPlanner.findBestExp via java
debug (see "Cheapest Plan:\n" [1]) or setting
org.eigenbase.relopt.RelOptPlanner log level to FINE

The generated java code can be captured via setting -Doptiq.debug=true. It
can be useful to check if Optiq performs excessive operations at the very
low level.

[1]:
https://github.com/apache/incubator-optiq/blob/master/core/src/main/java/org/eigenbase/relopt/volcano/VolcanoPlanner.java#L678


Vladimir