[jira] [Updated] (CALCITE-7422) Support large plan optimization mode for HepPlanner

["Wenzhuang Zhu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CALCITE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenzhuang Zhu updated CALCITE-7422:
-----------------------------------
    Summary: Support large plan optimization mode for HepPlanner  (was: Support large plan optimizaion mode for HepPlanner)

> Support large plan optimization mode for HepPlanner
> ---------------------------------------------------
>
>                 Key: CALCITE-7422
>                 URL: https://issues.apache.org/jira/browse/CALCITE-7422
>             Project: Calcite
>          Issue Type: Sub-task
>            Reporter: Wenzhuang Zhu
>            Priority: Major
>              Labels: pull-request-available
>
> This subtask includes the following items (controlled by isLargePlanMode):
> 1.HepPlanner supports graph reuse across different HepPrograms.
> Reason: Avoid calling setRoot multiple times when the user has multiple optimization phases. (HepSubProgram cannot be used because some global state must be changed between different HepPrograms.)
> 2.Fast graph GC for the subtree of a discarded vertex (discardedVertex).
> Reason: Avoid performing a full-graph traversal during garbage collection. Full graph GC is On. In the worst case, every transform in TOP_DOWN/BOTTOM_UP mode can trigger a full graph GC.
> 3.Use a large-plan-friendly graph iterator.
> Reason: DepthFirstIterator builds a list containing all nodes in the graph, which costs On memory and additional time to build the list. This is not suitable for large plans with 10K+ nodes. BreadthFirstIterator does not build such a list, and after firedRulesCache is enabled, BreadthFirstIterator and DepthFirstIterator fire the same number of rules per node. Therefore, for large-plan mode, using BreadthFirstIterator is generally better than DepthFirstIterator.
> 4.Recursivelly contract vertices.
> Reason: Large plans provide more opportunities for recursive contraction.
> Notes: A very deep plan may cause a stackoverflow when build the graph (recursive addVertex), but this is not the bottomneck. Compiler or other modules may need more stacks than optimizer, so we assume the -Xss is large enough.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)