You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Pranav Deshpande <de...@gmail.com> on 2022/08/26 20:29:15 UTC

Question Regarding Volcano Planner

Dear Apache Calcite Dev Team,
Sometimes my volcano planner fails to terminate because there are many
plans to search for (millions of them).

Is there a way to prune the search if a specific amount of time has elapsed
or if the same net cost is obtained for the Xth time, etc.

Requesting the community's guidance on this.

Thanks & Regards,
Pranav

Re: Question Regarding Volcano Planner

Posted by Roman Kondakov <ko...@mail.ru.INVALID>.
Hello Pranav,

you also can split your optimization process into several steps. For 
example you can add a heuristic phase (using HEP planner) before Volcano 
planning with filters pushdown. It is almost always a beneficial 
operation so you can avoid polluting search space with plans that are 
known to be not very good in advance.

You can find examples of optimization phases here [1]

[1] 
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/PlannerPhase.java

Thanks,

Roman

On 29.08.2022 03:54, Haisheng Yuan wrote:
> You need to first investigate whether the transformation rules or operators are correctly implemented or not. Typically there is a bug if you see the planner fails to terminate.
>
> There is cancelFlag variable and checkCancel() method in VolcanoPlanner.
> You can pass your own CancelFlag implementation to Context when constructing planner [1] so that you can interrupt the planning when it times out or the cost the greater or lower than some threshold.
>
> [1] https://github.com/apache/calcite/blob/e2f949d5d6cff79cbe8565bc3e85f437dee9fd7e/core/src/main/java/org/apache/calcite/plan/AbstractRelOptPlanner.java#L102
>
> On 2022/08/26 20:29:15 Pranav Deshpande wrote:
>> Dear Apache Calcite Dev Team,
>> Sometimes my volcano planner fails to terminate because there are many
>> plans to search for (millions of them).
>>
>> Is there a way to prune the search if a specific amount of time has elapsed
>> or if the same net cost is obtained for the Xth time, etc.
>>
>> Requesting the community's guidance on this.
>>
>> Thanks & Regards,
>> Pranav
>>

Re: Question Regarding Volcano Planner

Posted by Haisheng Yuan <hy...@apache.org>.
You need to first investigate whether the transformation rules or operators are correctly implemented or not. Typically there is a bug if you see the planner fails to terminate.

There is cancelFlag variable and checkCancel() method in VolcanoPlanner.
You can pass your own CancelFlag implementation to Context when constructing planner [1] so that you can interrupt the planning when it times out or the cost the greater or lower than some threshold.

[1] https://github.com/apache/calcite/blob/e2f949d5d6cff79cbe8565bc3e85f437dee9fd7e/core/src/main/java/org/apache/calcite/plan/AbstractRelOptPlanner.java#L102

On 2022/08/26 20:29:15 Pranav Deshpande wrote:
> Dear Apache Calcite Dev Team,
> Sometimes my volcano planner fails to terminate because there are many
> plans to search for (millions of them).
> 
> Is there a way to prune the search if a specific amount of time has elapsed
> or if the same net cost is obtained for the Xth time, etc.
> 
> Requesting the community's guidance on this.
> 
> Thanks & Regards,
> Pranav
>