You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "steven zhuang (Updated) (JIRA)" <ji...@apache.org> on 2012/03/02 03:46:59 UTC
[jira] [Updated] (HBASE-5505) if max file size set to 256M, Per
table hbase.hregion.max.filesize setting(at htable creation) overwritten by
value set in hbase-site.conf(which is not 256MB!)
[ https://issues.apache.org/jira/browse/HBASE-5505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
steven zhuang updated HBASE-5505:
---------------------------------
Description:
seems to me that when set to DEFAULT_MAX_FILE_SIZE, the max region file size set at Htable creation time may later be overwritten by value set in hbase-site.conf, i.e. no "Per table" hbase.hregion.max.filesize at all.
in this case, on Web UI in the table description, max file is still 256M, but we can find region file much bigger on HDFS.
In 0.92.0, o.a.h.hbase.regionserver.ConstantSizeRegionSplitPolicy, around line 33:
long maxFileSize = region.getTableDesc().getMaxFileSize();
// By default we split region if a file > HConstants.DEFAULT_MAX_FILE_SIZE.
if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) { //zx: here setting is over written by configuaration from hbase-site.conf
maxFileSize = getConf().getLong("hbase.hregion.max.filesize",
HConstants.DEFAULT_MAX_FILE_SIZE);
}
In 0.90.4(cdh3u3) or earlier version, this is found in o.a.h.hbase.regionserver.Store, around line 188:
long maxFileSize = info.getTableDesc().getMaxFileSize();
if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {
maxFileSize = conf.getLong("hbase.hregion.max.filesize",
HConstants.DEFAULT_MAX_FILE_SIZE);
}
was:
seems to me that when set as the DEFAULT_MAX_FILE_SIZE, the max region file size set at Htable creation time may later be overwritten by value set in hbase-site.conf, i.e. no "Per table" hbase.hregion.max.filesize at all.
in this case, on Web UI in the table description, max file is still 256M, but we can find region file much bigger on HDFS.
In 0.92.0, o.a.h.hbase.regionserver.ConstantSizeRegionSplitPolicy, around line 33:
long maxFileSize = region.getTableDesc().getMaxFileSize();
// By default we split region if a file > HConstants.DEFAULT_MAX_FILE_SIZE.
if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) { //zx: here setting is over written by configuaration from hbase-site.conf
maxFileSize = getConf().getLong("hbase.hregion.max.filesize",
HConstants.DEFAULT_MAX_FILE_SIZE);
}
In 0.90.4(cdh3u3) or earlier version, this is found in o.a.h.hbase.regionserver.Store, around line 188:
long maxFileSize = info.getTableDesc().getMaxFileSize();
if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {
maxFileSize = conf.getLong("hbase.hregion.max.filesize",
HConstants.DEFAULT_MAX_FILE_SIZE);
}
Summary: if max file size set to 256M, Per table hbase.hregion.max.filesize setting(at htable creation) overwritten by value set in hbase-site.conf(which is not 256MB!) (was: if max file size set to 256M, Per table hbase.hregion.max.filesize setting(at htable creation) overwritten by value set in hbase-site.conf(might not be 256MB!))
> if max file size set to 256M, Per table hbase.hregion.max.filesize setting(at htable creation) overwritten by value set in hbase-site.conf(which is not 256MB!)
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5505
> URL: https://issues.apache.org/jira/browse/HBASE-5505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.90.4, 0.92.0
> Reporter: steven zhuang
>
> seems to me that when set to DEFAULT_MAX_FILE_SIZE, the max region file size set at Htable creation time may later be overwritten by value set in hbase-site.conf, i.e. no "Per table" hbase.hregion.max.filesize at all.
> in this case, on Web UI in the table description, max file is still 256M, but we can find region file much bigger on HDFS.
> In 0.92.0, o.a.h.hbase.regionserver.ConstantSizeRegionSplitPolicy, around line 33:
> long maxFileSize = region.getTableDesc().getMaxFileSize();
> // By default we split region if a file > HConstants.DEFAULT_MAX_FILE_SIZE.
> if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) { //zx: here setting is over written by configuaration from hbase-site.conf
> maxFileSize = getConf().getLong("hbase.hregion.max.filesize",
> HConstants.DEFAULT_MAX_FILE_SIZE);
> }
> In 0.90.4(cdh3u3) or earlier version, this is found in o.a.h.hbase.regionserver.Store, around line 188:
> long maxFileSize = info.getTableDesc().getMaxFileSize();
> if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {
> maxFileSize = conf.getLong("hbase.hregion.max.filesize",
> HConstants.DEFAULT_MAX_FILE_SIZE);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira