[jira] [Comment Edited] (HBASE-11747) ClusterStatus is too bulky

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097392#comment-14097392 ] 

Andrew Purtell edited comment on HBASE-11747 at 8/14/14 7:05 PM:
-----------------------------------------------------------------

We can add *new* APIs. I wonder if it would be workable to introduce a streaming status API where the client uses a cursor to iterate over the master's picture of the cluster. Might be tricky wherever regions have migrated or servers have come and gone. The master would have to provide either a consistent snapshot of state or track changes since the client opened the cursor and mix in change deltas with iteration results.


was (Author: apurtell):
We can add *new* APIs. I wonder if it would be workable to introduce a streaming status API where the client uses a cursor to iterate over the master's picture of the cluster. Might be tricky wherever regions have migrated or servers have come and gone. The master would have to provide either a consistent snapshot of state or track changes since the client opened the curser and mix in change deltas with iteration results.

> ClusterStatus is too bulky 
> ---------------------------
>
>                 Key: HBASE-11747
>                 URL: https://issues.apache.org/jira/browse/HBASE-11747
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Virag Kothari
>         Attachments: exceptiontrace
>
>
> Following exception on 0.98 with 1M regions on cluster with 160 region servers
> {code}
> Caused by: java.io.IOException: Call to regionserverhost:port failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large.  May be malicious.  Use CodedInputStream.setSizeLimit() to increase the size limit.
> 	at org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
> 	at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
> 	at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
> 	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
> 	at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
> 	... 43 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large.  May be malicious.  Use CodedInputStream.setSizeLimit() to increase the size limit.
> 	at com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)