You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Harsh J (JIRA)" <ji...@apache.org> on 2012/11/02 15:11:11 UTC
[jira] [Created] (HBASE-7084) Raise default minimum region split
size
Harsh J created HBASE-7084:
------------------------------
Summary: Raise default minimum region split size
Key: HBASE-7084
URL: https://issues.apache.org/jira/browse/HBASE-7084
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.96.0
Reporter: Harsh J
Priority: Trivial
Several times I've seen folks mentioning to raise region split sizes from the default 256m to something more suitable (mostly folks say 1 GB, sometimes 2 GB, more than other options), in order to control the # of regions explosion once they begin heavy-hitting the tables.
Perhaps it makes sense to raise the default itself, since there are ways to bring it down per-table if needed by some use-cases?
Opening this as a discussion first, since this "too many regions" trouble is quite prevalent among newcomers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7084) Raise default minimum region split
size
Posted by "Kevin Odell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489438#comment-13489438 ]
Kevin Odell commented on HBASE-7084:
------------------------------------
use HFilev2 sounds weird. I mean for our region starting point we don't have to worry about HFilev1. For larger environments running .92+ we usually recommend between 10 - 20GB to keep the region count down.
> Raise default minimum region split size
> ---------------------------------------
>
> Key: HBASE-7084
> URL: https://issues.apache.org/jira/browse/HBASE-7084
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: Harsh J
> Priority: Trivial
>
> Several times I've seen folks mentioning to raise region split sizes from the default 256m to something more suitable (mostly folks say 1 GB, sometimes 2 GB, more than other options), in order to control the # of regions explosion once they begin heavy-hitting the tables.
> Perhaps it makes sense to raise the default itself, since there are ways to bring it down per-table if needed by some use-cases?
> Opening this as a discussion first, since this "too many regions" trouble is quite prevalent among newcomers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7084) Raise default minimum region split
size
Posted by "Kevin Odell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489436#comment-13489436 ]
Kevin Odell commented on HBASE-7084:
------------------------------------
+1, I think with .90 being an exception as opposed to a rule these days we should use HFilev2 and set the default to something like 2GB and then have our documentation discuss splitting for smaller workloads. It is much easier to split then it is to merge.
> Raise default minimum region split size
> ---------------------------------------
>
> Key: HBASE-7084
> URL: https://issues.apache.org/jira/browse/HBASE-7084
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: Harsh J
> Priority: Trivial
>
> Several times I've seen folks mentioning to raise region split sizes from the default 256m to something more suitable (mostly folks say 1 GB, sometimes 2 GB, more than other options), in order to control the # of regions explosion once they begin heavy-hitting the tables.
> Perhaps it makes sense to raise the default itself, since there are ways to bring it down per-table if needed by some use-cases?
> Opening this as a discussion first, since this "too many regions" trouble is quite prevalent among newcomers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7084) Raise default minimum region split
size
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489442#comment-13489442 ]
Harsh J commented on HBASE-7084:
--------------------------------
Thanks Kevin! I wouldn't go so high though for defaults, since there are users who don't mind a few tens or a hundred of regions of a single table per RS, just for parallelism (over fragmented files - better availability in face of per-region failure, if any), and of course - that is also data size dependent.
> Raise default minimum region split size
> ---------------------------------------
>
> Key: HBASE-7084
> URL: https://issues.apache.org/jira/browse/HBASE-7084
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: Harsh J
> Priority: Trivial
>
> Several times I've seen folks mentioning to raise region split sizes from the default 256m to something more suitable (mostly folks say 1 GB, sometimes 2 GB, more than other options), in order to control the # of regions explosion once they begin heavy-hitting the tables.
> Perhaps it makes sense to raise the default itself, since there are ways to bring it down per-table if needed by some use-cases?
> Opening this as a discussion first, since this "too many regions" trouble is quite prevalent among newcomers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7084) Raise default minimum region split
size
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489451#comment-13489451 ]
Jean-Daniel Cryans commented on HBASE-7084:
-------------------------------------------
Take a look at HBASE-4365, this is mostly already done.
> Raise default minimum region split size
> ---------------------------------------
>
> Key: HBASE-7084
> URL: https://issues.apache.org/jira/browse/HBASE-7084
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: Harsh J
> Priority: Trivial
>
> Several times I've seen folks mentioning to raise region split sizes from the default 256m to something more suitable (mostly folks say 1 GB, sometimes 2 GB, more than other options), in order to control the # of regions explosion once they begin heavy-hitting the tables.
> Perhaps it makes sense to raise the default itself, since there are ways to bring it down per-table if needed by some use-cases?
> Opening this as a discussion first, since this "too many regions" trouble is quite prevalent among newcomers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7084) Raise default minimum region split
size
Posted by "Kevin Odell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489443#comment-13489443 ]
Kevin Odell commented on HBASE-7084:
------------------------------------
You think 1GB would be a better starting point? It would save us 4x regions on the runaway tables.
> Raise default minimum region split size
> ---------------------------------------
>
> Key: HBASE-7084
> URL: https://issues.apache.org/jira/browse/HBASE-7084
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: Harsh J
> Priority: Trivial
>
> Several times I've seen folks mentioning to raise region split sizes from the default 256m to something more suitable (mostly folks say 1 GB, sometimes 2 GB, more than other options), in order to control the # of regions explosion once they begin heavy-hitting the tables.
> Perhaps it makes sense to raise the default itself, since there are ways to bring it down per-table if needed by some use-cases?
> Opening this as a discussion first, since this "too many regions" trouble is quite prevalent among newcomers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira