You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/02/20 23:55:00 UTC
[jira] [Commented] (IMPALA-8690) Better eviction algorithm for data
cache
[ https://issues.apache.org/jira/browse/IMPALA-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041377#comment-17041377 ]
ASF subversion and git services commented on IMPALA-8690:
---------------------------------------------------------
Commit eead1873dddd2ba2b2d97aeb80482835e3e1c7eb in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=eead187 ]
IMPALA-8690 (prep 1): Copy Kudu cache code to be/src/util/cache
The data cache uses Kudu's cache code in be/src/kudu/util.
To make it easy to make Impala specific changes (such as
introducing new cache eviction algorithms), this copies the
code into Impala's utilities directory.
To make the review easier, this is a simple copy of the files
with a minimal CMakeLists.txt and RAT changes. This does not link
the resulting library into anything.
A subsequent change will modify the code to be usable by
Impala. To test, I verified that this compiles.
Change-Id: I0e7b8b852b56613803dea125456fa9e8b3736d76
Reviewed-on: http://gerrit.cloudera.org:8080/15169
Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
Tested-by: Joe McDonnell <jo...@cloudera.com>
> Better eviction algorithm for data cache
> ----------------------------------------
>
> Key: IMPALA-8690
> URL: https://issues.apache.org/jira/browse/IMPALA-8690
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.3.0
> Reporter: Michael Ho
> Assignee: Joe McDonnell
> Priority: Major
>
> With the current implementation of data cache, all data access will be cached regardless of the access pattern. The current LRU eviction algorithm is not resistant to scan traffic so in case some users scan a big fact table, a lot of the heavily accessed items will be evicted inevitably. We should adopt better eviction algorithm (e.g. LRFU or some other well known ones in the literature). Would be nice to evaluate it against some users' traces now that IMPALA-8542 is fixed.
> In the short run, we probably need some workaround (e.g. query hints to disable caching for certain tables). Will file a separate jira for it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org