[jira] [Resolved] (CALCITE-7422) Support large plan optimization mode for HepPlanner

["Alessandro Solimando (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CALCITE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alessandro Solimando resolved CALCITE-7422.
-------------------------------------------
    Fix Version/s: 1.42.0
       Resolution: Fixed

Fixed via [{{f1f9a2b}}|https://github.com/apache/calcite/commit/f1f9a2b1e3c25188c3493068bd768aadfad29fd8], thank you [~wenzhuang.zwz] for your contribution and thanks to [~mbudiu] and [~jensen] for your reviews!

> Support large plan optimization mode for HepPlanner
> ---------------------------------------------------
>
>                 Key: CALCITE-7422
>                 URL: https://issues.apache.org/jira/browse/CALCITE-7422
>             Project: Calcite
>          Issue Type: Sub-task
>            Reporter: Wenzhuang Zhu
>            Assignee: Wenzhuang Zhu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.42.0
>
>
> This subtask includes the following items (controlled by isLargePlanMode):
> 1.HepPlanner supports graph reuse across different HepPrograms.
> Reason: Avoid calling setRoot multiple times when the user has multiple optimization phases. (HepSubProgram cannot be used because some global state must be changed between different HepPrograms.)
> 2.Fast graph GC for the subtree of a discarded vertex (discardedVertex).
> Reason: Avoid performing a full-graph traversal during garbage collection. Full graph GC is On. In the worst case, every transform in TOP_DOWN/BOTTOM_UP mode can trigger a full graph GC.
> 3.Use a large-plan-friendly graph iterator.
> Reason: DepthFirstIterator builds a list containing all nodes in the graph, which costs On memory and additional time to build the list. This is not suitable for large plans with 10K+ nodes. BreadthFirstIterator does not build such a list, and after firedRulesCache is enabled, BreadthFirstIterator and DepthFirstIterator fire the same number of rules per node. Therefore, for large-plan mode, using BreadthFirstIterator is generally better than DepthFirstIterator.
> 4.Recursivelly contract vertices.
> Reason: Large plans provide more opportunities for recursive contraction.
> Notes: A very deep plan may cause a stackoverflow when build the graph (recursive addVertex), but this is not the bottomneck. Compiler or other modules may need more stacks than optimizer, so we assume the -Xss is large enough.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)