[jira] [Commented] (QUICKSTEP-60) Delay memory allocation for hash tables

["Julian Hyde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/QUICKSTEP-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609048#comment-15609048 ] 

Julian Hyde commented on QUICKSTEP-60:
--------------------------------------

I find it best to aim for a complete separation between plan and runtime state. If you do this your plan is re-entrant: it can be in use by many queries in parallel, re-executed (maybe changing bind variables) without re-planning, and you can have multiple instances of each operator (e.g. owned by different threads). I was at Oracle when they did this re-engineering in the 90's. It was painful. Get it right early.

> Delay memory allocation for hash tables
> ---------------------------------------
>
>                 Key: QUICKSTEP-60
>                 URL: https://issues.apache.org/jira/browse/QUICKSTEP-60
>             Project: Apache Quickstep
>          Issue Type: Improvement
>          Components: Query Execution, Query Optimizer, Relational Operators
>            Reporter: Harshad Deshmukh
>            Assignee: Harshad Deshmukh
>              Labels: performance
>
> Currently, Quickstep allocates memory for the hash tables during the query execution plan generation (specifically in ExecutionGenerator). The memory allocation results into memory reservation in the buffer pool. In some cases (e.g. TPC-H Q21), the estimated memory requirements for the hash tables in the query plan is huge. This means during the query execution there is an artificial shortage of memory. This degrades the performance, as it leads to evictions. 
> To avoid this issue, the query scheduler should allocate memory for the hash tables as and when it is needed, during the query execution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)