You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2013/01/01 16:06:12 UTC
[jira] [Commented] (HBASE-7404) Bucket Cache:A solution about
CMS,Heap Fragment and Big Cache on HBASE
[ https://issues.apache.org/jira/browse/HBASE-7404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541645#comment-13541645 ]
Ted Yu commented on HBASE-7404:
-------------------------------
For patch v9, I got the following compilation error:
{code}
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hbase-server: Compilation failure
[ERROR] /Users/tyu/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java:[303,30] illegal forward reference
{code}
I think this is due to deserializerIdentifier being referenced in the CacheableDeserializer:
{code}
private static final CacheableDeserializer<Cacheable> blockDeserializer =
...
+ @Override
+ public int getDeserialiserIdentifier() {
+ return deserializerIdentifier;
+ }
...
+ private static final int deserializerIdentifier = CacheableDeserializerIdManager
+ .registerDeserializer(blockDeserializer);
{code}
> Bucket Cache:A solution about CMS,Heap Fragment and Big Cache on HBASE
> ----------------------------------------------------------------------
>
> Key: HBASE-7404
> URL: https://issues.apache.org/jira/browse/HBASE-7404
> Project: HBase
> Issue Type: New Feature
> Affects Versions: 0.94.3
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.5
>
> Attachments: BucketCache.pdf, hbase-7404-94v2.patch, hbase-7404-trunkv2.patch, hbase-7404-trunkv9.patch, Introduction of Bucket Cache.pdf
>
>
> First, thanks @neil from Fusion-IO share the source code.
> What's Bucket Cache?
> It could greatly decrease CMS and heap fragment by GC
> It support a large cache space for High Read Performance by using high speed disk like Fusion-io
> 1.An implementation of block cache like LruBlockCache
> 2.Self manage blocks' storage position through Bucket Allocator
> 3.The cached blocks could be stored in the memory or file system
> 4.Bucket Cache could be used as a mainly block cache(see CombinedBlockCache), combined with LruBlockCache to decrease CMS and fragment by GC.
> 5.BucketCache also could be used as a secondary cache(e.g. using Fusionio to store block) to enlarge cache space
> How about SlabCache?
> We have studied and test SlabCache first, but the result is bad, because:
> 1.SlabCache use SingleSizeCache, its use ratio of memory is low because kinds of block size, especially using DataBlockEncoding
> 2.SlabCache is uesd in DoubleBlockCache, block is cached both in SlabCache and LruBlockCache, put the block to LruBlockCache again if hit in SlabCache , it causes CMS and heap fragment don't get any better
> 3.Direct heap performance is not good as heap, and maybe cause OOM, so we recommend using "heap" engine
> See more in the attachment and in the patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira