You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/01/06 14:00:00 UTC
[jira] [Commented] (IMPALA-11470) Support Caching Codegen Functions
[ https://issues.apache.org/jira/browse/IMPALA-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655459#comment-17655459 ]
ASF subversion and git services commented on IMPALA-11470:
----------------------------------------------------------
Commit 839a25c89b5620e8bdc37383c8512d912fcb5d0f in impala's branch refs/heads/master from Yida Wu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=839a25c89 ]
IMPALA-11786: Preserve memory for codegen cache
IMPALA-11470 adds support for codegen cache, however the admission
controller is not aware of the memory usage of the codegen cache,
while the codegen cache is actually using the memory quota from
the query memory. It could result in query failures when running
heavy workloads and admission controller has fully admitted queries.
This patch subtracts the codegen cache capacity from the admission
memory limit during initialization, therefore preserving the memory
consumption of codegen cache from the beginning, and treating it as
a separate memory independent to the query memory reservation.
Also reduces the max codegen cache memory from 20 percent to 10
percent, and changes some failed testcases due to the reduction of
the admit memory limit.
Tests:
Passed exhaustive tests.
Change-Id: Iebdc04ba1b91578d74684209a11c815225b8505a
Reviewed-on: http://gerrit.cloudera.org:8080/19377
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Support Caching Codegen Functions
> ---------------------------------
>
> Key: IMPALA-11470
> URL: https://issues.apache.org/jira/browse/IMPALA-11470
> Project: IMPALA
> Issue Type: Task
> Components: Backend
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
> Fix For: Impala 4.3.0
>
>
> Impala traditionally has been a leader in low latency queries. However, it is reported that for sub second queries codegen could take the majority of the time, for example, codegen alone can take 300ms for a sub second query.
>
> This task aims to add support of memory caches for codegen functions to improve the performance of sub second queries by reducing the codegen generation time. Users can specify the total cache size for all of the codegen functions and which query is going to use the cache.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org