[jira] [Commented] (DRILL-8529) Caching QueryPlan Results

["Vincent de Gans (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019583#comment-18019583 ] 

Vincent de Gans commented on DRILL-8529:
----------------------------------------

[~cgivre] Thank you for the suggestions and examples. I have created a draft pull request at https://github.com/apache/drill/pull/3023 containing a rough implementation of the caching mechanism with some settings to tweak the ttl and cache size. I am still unsure how to properly test this feature. Any and all feedback is always welcome, as I am still learning about the project and would like to improve as a programmer.

> Caching QueryPlan Results
> -------------------------
>
>                 Key: DRILL-8529
>                 URL: https://issues.apache.org/jira/browse/DRILL-8529
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Query Planning & Optimization
>            Reporter: Vincent de Gans
>            Priority: Minor
>             Fix For: Future
>
>
> I propose introducing a caching mechanism for the output of `getQueryPlan()` in cases where:
> - The input SQL query is the same as a previously seen one
> - The schema or relevant metadata used for planning has not changed
> - The cached result has not expired, based on a configurable time-to-live (TTL)
> Proposed Caching Features
> - Toggle to enable or disable query plan caching
> - Configurable TTL-based invalidation
> - Optional schema metadata verification to detect changes in underlying data sources
> Motivation
> - Reduce planner overhead for repeated queries
> - Improve response times in environments where query plans are reused
> - Provide an optional optimization that can be enabled when beneficial



--
This message was sent by Atlassian Jira
(v8.20.10#820010)