You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Kyle Purtell (Jira)" <ji...@apache.org> on 2024/04/10 18:46:00 UTC
[jira] [Comment Edited] (HBASE-28447) New configuration to override the hfile specific blocksize
[ https://issues.apache.org/jira/browse/HBASE-28447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835852#comment-17835852 ]
Andrew Kyle Purtell edited comment on HBASE-28447 at 4/10/24 6:45 PM:
----------------------------------------------------------------------
PR #5820 introduces a new configuration setting - "hfile.block.size" - that, if set, will define the default blocksize to use when writing HFiles if a column family schema does not define its own non-default block size. This is a bit complicated but required for compatability. The rules are:
* If the schema specifies a non default block size, use it.
* Otherwise, if the configuration specifies a non default block size, use it.
* Otherwise, use the default block size.
The default is defined by HConstants.DEFAULT_BLOCKSIZE.
was (Author: apurtell):
PR #5820 introduces a new configuration setting - "hfile.block.size" - that, if set, will define the default blocksize to use when writing HFiles if a column family schema does not define its own non-default block size. This is a bit complicated but required for compatability. The rules are:
* If the schema specifies a non default block size, use it.
* Otherwise, if the configuration specifies a non default block size, use it.
* Otherwise, use the default block size. The default is defined by HConstants.DEFAULT_BLOCKSIZE.
> New configuration to override the hfile specific blocksize
> ----------------------------------------------------------
>
> Key: HBASE-28447
> URL: https://issues.apache.org/jira/browse/HBASE-28447
> Project: HBase
> Issue Type: Improvement
> Reporter: Gourab Taparia
> Assignee: Andrew Kyle Purtell
> Priority: Minor
> Labels: pull-request-available
> Fix For: 2.6.0, 2.7.0, 3.0.0-beta-2, 2.5.9
>
>
> Right now there is no config attached to the HFile block size by which we can override the default. The default is set to 64 KB in HConstants.DEFAULT_BLOCKSIZE . We need a global config property that would go on hbase-site.xm which can control this value.
> Since the BLOCKSIZE is tracked at the column family level - we will need to respect the CFD value first. Also, configuration settings are also something that can be set in schema, at the column or table level, and will override the relevant values from the site file. Below is the precedence order we can use to get the final blocksize value :
> {code:java}
> ColumnFamilyDescriptor.BLOCKSIZE > schema level site configuration overrides > site configuration > HConstants.DEFAULT_BLOCKSIZE{code}
> PS: There is one related config “hbase.mapreduce.hfileoutputformat.blocksize” however that is specific to map-reduce jobs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)