You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2014/05/21 01:24:39 UTC
[jira] [Comment Edited] (HBASE-11165) Scaling so cluster can host
1M regions and beyond (50M regions?)
[ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004107#comment-14004107 ]
Andrew Purtell edited comment on HBASE-11165 at 5/20/14 11:23 PM:
------------------------------------------------------------------
Looks like how the master uses ZooKeeper to manage assignments will need updating. I created a table with 1M regions using HBASE-11211. After creating all 1M regions, the master attempted IO of an enormous znode and aborted:
{noformat}
2014-05-20 16:07:40,402 WARN [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session 0x1461be2f6220000 for server localhost/127.0.0.1:2181, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Packet len6378500 is out of range!
at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2014-05-20 16:07:40,506 WARN [master:localhost:8100] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/region-in-transition
...
2014-05-20 16:07:56,754 INFO [master:localhost:8100] master.HMaster: Aborting
{noformat}
was (Author: apurtell):
Looks like how the master uses ZooKeeper to manage assignments will need updating. I created a table with 1M regions using HBASE-11211. After creating all 1M regions, the master attempted a write of an enormous znode and aborted when ZK rejected it:
{noformat}
2014-05-20 16:07:40,402 WARN [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session 0x1461be2f6220000 for server localhost/127.0.0.1:2181, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Packet len6378500 is out of range!
at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2014-05-20 16:07:40,506 WARN [master:localhost:8100] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/region-in-transition
...
2014-05-20 16:07:56,754 INFO [master:localhost:8100] master.HMaster: Aborting
{noformat}
> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
> Key: HBASE-11165
> URL: https://issues.apache.org/jira/browse/HBASE-11165
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569" and comments on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe even 50M later. This issue is about discussing how we will do that (or if not 50M on a cluster, how otherwise we can attain same end).
> More detail to follow.
--
This message was sent by Atlassian JIRA
(v6.2#6252)