[jira] [Commented] (KUDU-613) Scan-resistant cache replacement algorithm for the block cache

["Andrew Wong (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247540#comment-17247540 ] 

Andrew Wong commented on KUDU-613:
----------------------------------

The Apache Impala project has pulled in Kudu's block cache implementation and [extended it with LIRS|https://gerrit.cloudera.org/c/15306/]. It's probably worth pulling those bits in and seeing how they fare against contentious large-scan workloads in Kudu.

LIRS: [http://web.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-02-6.pdf]

 

> Scan-resistant cache replacement algorithm for the block cache
> --------------------------------------------------------------
>
>                 Key: KUDU-613
>                 URL: https://issues.apache.org/jira/browse/KUDU-613
>             Project: Kudu
>          Issue Type: Improvement
>          Components: perf
>    Affects Versions: M4.5
>            Reporter: Andrew Wang
>            Priority: Major
>              Labels: roadmap-candidate
>
> The block cache currently uses LRU, which is vulnerable to large scan workloads. It'd be good to implement something like 2Q.
> ARC (patent encumbered, but good for ideas):
> https://www.usenix.org/conference/fast-03/arc-self-tuning-low-overhead-replacement-cache
> HBase (2Q like):
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java



--
This message was sent by Atlassian Jira
(v8.3.4#803005)