You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ming Ma (Created) (JIRA)" <ji...@apache.org> on 2011/10/01 00:33:50 UTC
[jira] [Created] (HBASE-4524) support for more than one region in
.META. table
support for more than one region in .META. table
------------------------------------------------
Key: HBASE-4524
URL: https://issues.apache.org/jira/browse/HBASE-4524
Project: HBase
Issue Type: Improvement
Reporter: Ming Ma
Assignee: Ming Ma
It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
1) .META. table update with user region info.
2) .META. regions assignment.
3) .META. table split handling.
Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4524) support for more than one region in
.META. table
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118545#comment-13118545 ]
stack commented on HBASE-4524:
------------------------------
Reasons to let meta split other than because the cluster has millions of regions:
+ Distribute the load on meta.
+ Its 'catastrophic' if server carrying meta goes away. It'd be *less* catastrophic if the regionserver were carrying only a piece of meta.
> support for more than one region in .META. table
> ------------------------------------------------
>
> Key: HBASE-4524
> URL: https://issues.apache.org/jira/browse/HBASE-4524
> Project: HBase
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
>
> It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
> 1) .META. table update with user region info.
> 2) .META. regions assignment.
> 3) .META. table split handling.
> Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4524) support for more than one region in
.META. table
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118652#comment-13118652 ]
stack commented on HBASE-4524:
------------------------------
bq. add a feature so that the load balancer can be configured to keep META "alone" with no other regions on the server. This would be used on big clusters.
When this server crashed, another server would have to be cleared of its regions to dedicate to meta. There'd be some churn. Log splitting should be fast though.
Changing the balancer would be less intrusive than, for example, having hlog write two logs, one for user space regions and another for meta edits.
bq. remove ROOT, and just store META directly in ZK (ROOT is essentially vestigial these days)
I'm +1 on killing root. If we had to have multiple regions in meta table, we could store all locations in zk and then have client sort them whenever it read the list of metas from zk.
On metadata size, 0.92 shrinks significantly the size per region entry because we remove the table descriptor per region entry. We can shrink the meta entry size further if needs be. That said, I can imagine we'll start to add data per region (e.g. your region history might be one such scenario).
On 1M regions on a cluster, that was the fashion once. Tendency seems to be toward less bigger regions per server now though as you say Ming, 10k regionservers makes for a 1M w/ only 100 regions per server. But 1M rows is not that much though. You could probably squeeze 1M rows each with a few versions each into a 256M single region I'd say (Haven't checked). I'd doubt this will be the primary obstacle in the way of our getting to 10k cluster size.
> support for more than one region in .META. table
> ------------------------------------------------
>
> Key: HBASE-4524
> URL: https://issues.apache.org/jira/browse/HBASE-4524
> Project: HBase
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
>
> It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
> 1) .META. table update with user region info.
> 2) .META. regions assignment.
> 3) .META. table split handling.
> Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4524) support for more than one region in
.META. table
Posted by "Jonathan Gray (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119745#comment-13119745 ]
Jonathan Gray commented on HBASE-4524:
--------------------------------------
+1 removing root and putting META location into zk.
-1 on thinking about splittable META right now.
+1 thinking about mirroring META in ZK, adding new locations/redirects in NSREs, and other optimizations and availability improvements
> support for more than one region in .META. table
> ------------------------------------------------
>
> Key: HBASE-4524
> URL: https://issues.apache.org/jira/browse/HBASE-4524
> Project: HBase
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
>
> It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
> 1) .META. table update with user region info.
> 2) .META. regions assignment.
> 3) .META. table split handling.
> Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4524) support for more than one region in
.META. table
Posted by "Ming Ma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118603#comment-13118603 ]
Ming Ma commented on HBASE-4524:
--------------------------------
Agree on the "remove ROOT" suggestion for the current scale. I had this question when fixing HBASE-4455, why do we need such redirection in the current scale?
Does anyone know if 1M regions is a realistic scenario any time soon? 1M regions implies 10K RSs if we have 100 regions per RS. That is a really large scale HBase cluster.
Also, is there a chance we will put lot more meta data for each region to .META. table, thus we don't need 1M regions to make .META. table large? If we somehow need to put 100K bytes for each region in .META. table, we just need 100K regions to make .META. table reach 10GB.
> support for more than one region in .META. table
> ------------------------------------------------
>
> Key: HBASE-4524
> URL: https://issues.apache.org/jira/browse/HBASE-4524
> Project: HBase
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
>
> It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
> 1) .META. table update with user region info.
> 2) .META. regions assignment.
> 3) .META. table split handling.
> Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4524) support for more than one region in
.META. table
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118590#comment-13118590 ]
Todd Lipcon commented on HBASE-4524:
------------------------------------
I've spoken to some Bigtable engineers who said that the two-layer ROOT/META thing turned out to be a bit of a mistake given today's machines. The reason being that today's servers, if you put only META on a single server, easily have enough RAM and CPU horsepower to handle a several-GB meta region and hundreds of thousands of requests/second, since all the requests can hit cache.
The recovery argument makes some sense, but I think if you deploy META alone on a server and configure it to flush often, the log recovery should be near-instantaneous and failover would be fast. If META is split across many regions, then it's likely to be intermingled on the same servers as user data, and hence would be slow to split logs.
So I'm not against doing this, but another alternate path might be:
- add a feature so that the load balancer can be configured to keep META "alone" with no other regions on the server. This would be used on big clusters.
- remove ROOT, and just store META directly in ZK (ROOT is essentially vestigial these days)
Any thoughts on this? I'd just like to consider the option of simplifying the META/ROOT situation rather than complicating it.
> support for more than one region in .META. table
> ------------------------------------------------
>
> Key: HBASE-4524
> URL: https://issues.apache.org/jira/browse/HBASE-4524
> Project: HBase
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
>
> It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
> 1) .META. table update with user region info.
> 2) .META. regions assignment.
> 3) .META. table split handling.
> Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4524) support for more than one region in
.META. table
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118613#comment-13118613 ]
Todd Lipcon commented on HBASE-4524:
------------------------------------
My opinion is that we shouldn't necessarily design for the 10,000 node cluster yet :) We'll hit other scalability issues somewhere along the line with that, too.
bq. If we somehow need to put 100K bytes for each region in .META. table
Why would we need so much data per region? 1K seems much more reasonable.
> support for more than one region in .META. table
> ------------------------------------------------
>
> Key: HBASE-4524
> URL: https://issues.apache.org/jira/browse/HBASE-4524
> Project: HBase
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
>
> It seems there are some assumptions in the code that .META. table only has one region FIRST_META_REGIONINFO in the following areas:
> 1) .META. table update with user region info.
> 2) .META. regions assignment.
> 3) .META. table split handling.
> Perhaps we don't have such requirement until we scale to really large number of regions like 1M.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira