You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Anoop Sam John (JIRA)" <ji...@apache.org> on 2017/07/20 06:16:00 UTC
[jira] [Updated] (HBASE-15477) Do not save 'next block header' when
we cache hfileblocks
[ https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anoop Sam John updated HBASE-15477:
-----------------------------------
Release Note:
Fix over-persisting in blockcache; no longer save the block PLUS the header of the next block (33 bytes) when writing the cache.
Also removes support for hfileblock v1; hfile block v1 was used writing hfile v1. hfile v1 was the default in hbase before hbase-0.92. hbase.96 would not start unless all v1 hfiles had been compacted out of the cluster.
was:
Fix over-persisting in blockcache; no longer save the block PLUS the header of the next block (3 bytes) when writing the cache.
Also removes support for hfileblock v1; hfile block v1 was used writing hfile v1. hfile v1 was the default in hbase before hbase-0.92. hbase.96 would not start unless all v1 hfiles had been compacted out of the cluster.
> Do not save 'next block header' when we cache hfileblocks
> ---------------------------------------------------------
>
> Key: HBASE-15477
> URL: https://issues.apache.org/jira/browse/HBASE-15477
> Project: HBase
> Issue Type: Sub-task
> Components: BlockCache, Performance
> Reporter: stack
> Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 15366v4.patch, 15477.backport.branch-1.patch, 15477.backport.branch-1.v2.patch, 15477.backport.branch-1.v3.patch, 15477.backport.branch-1.v4.patch, 15477.backport.branch-1.v4.patch, 15477.backport.branch-1.v5.patch, 15477.backport.branch-1.v5.patch, 15477.backport.branch-1.v6.patch, 15477.backport.branch-1.v7.patch, 15477.patch, 15477v2.patch, 15477v3.patch, 15477v3.patch, 15477v4.patch
>
>
> When we read from HDFS, we overread to pick up the next blocks header.
> Doing this saves a seek as we move through the hfile; we save having to
> do an explicit seek just to read the block header every time we need to
> read the body. We used to read in the next header as part of the
> current blocks buffer. This buffer was then what got persisted to
> blockcache; so we were over-persisting wrtiting out our block plus the
> next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
> complicated by this extra tail. Fix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)