You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2019/03/27 00:54:00 UTC
[jira] [Updated] (HBASE-22114) Port HBASE-15560 (TinyLFU-based
BlockCache) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-22114:
-----------------------------------
Description:
HBASE-15560 introduces the TinyLFU cache policy for the blockcache.
W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the frequency in a counting sketch, ages periodically by halving the counters, and orders entries by SLRU. An entry is discarded by comparing the frequency of the new arrival (candidate) to the SLRU's victim, and keeping the one with the highest frequency. This allows the operations to be performed in O(1) time and, though the use of a compact sketch, a much larger history is retained beyond the current working set. In a variety of real world traces the policy had [near optimal hit rates|https://github.com/ben-manes/caffeine/wiki/Efficiency].
The implementation of HBASE-15560 uses several Java 8 idioms, depends on JRE 8+ type Optional, and has dependencies on libraries compiled with Java 8+ bytecode. It could be backported to branch-1 but must be made optional both at compile time and runtime, enabled by the 'build-with-jdk8' build profile.
The TinyLFU policy must go into its own build module.
The blockcache must be modified to load L1 (and perhaps L2) implementation/policy dynamically at startup by reflection, with implementation classname specified in site configuration. This modification must be forward ported to maintain configuration sanity among the branches.
was:
HBASE-15560 introduces the TinyLFU cache policy for the blockcache.
W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the frequency in a counting sketch, ages periodically by halving the counters, and orders entries by SLRU. An entry is discarded by comparing the frequency of the new arrival (candidate) to the SLRU's victim, and keeping the one with the highest frequency. This allows the operations to be performed in O(1) time and, though the use of a compact sketch, a much larger history is retained beyond the current working set. In a variety of real world traces the policy had [near optimal hit rates|https://github.com/ben-manes/caffeine/wiki/Efficiency].
The implementation of HBASE-15560 uses several Java 8 idioms, depends on JRE 8+ type Optional, and has dependencies on libraries compiled with Java 8+ bytecode. It could be backported to branch-1 but must be made optional both at compile time and runtime, enabled by the 'build-with-jdk8' build profile. To this end the blockcache should be slightly modified to load L1 (and perhaps L2) implementation/policy dynamically at startup by reflection, with implementation classname specified in site configuration. This modification should be forward ported to maintain configuration sanity among the branches.
> Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1
> -------------------------------------------------------
>
> Key: HBASE-22114
> URL: https://issues.apache.org/jira/browse/HBASE-22114
> Project: HBase
> Issue Type: Sub-task
> Reporter: Andrew Purtell
> Priority: Major
> Fix For: 1.6.0
>
>
> HBASE-15560 introduces the TinyLFU cache policy for the blockcache.
> W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the frequency in a counting sketch, ages periodically by halving the counters, and orders entries by SLRU. An entry is discarded by comparing the frequency of the new arrival (candidate) to the SLRU's victim, and keeping the one with the highest frequency. This allows the operations to be performed in O(1) time and, though the use of a compact sketch, a much larger history is retained beyond the current working set. In a variety of real world traces the policy had [near optimal hit rates|https://github.com/ben-manes/caffeine/wiki/Efficiency].
> The implementation of HBASE-15560 uses several Java 8 idioms, depends on JRE 8+ type Optional, and has dependencies on libraries compiled with Java 8+ bytecode. It could be backported to branch-1 but must be made optional both at compile time and runtime, enabled by the 'build-with-jdk8' build profile.
> The TinyLFU policy must go into its own build module.
> The blockcache must be modified to load L1 (and perhaps L2) implementation/policy dynamically at startup by reflection, with implementation classname specified in site configuration. This modification must be forward ported to maintain configuration sanity among the branches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)