Error while doing multi get from HBase

[anand nalya <a....@computer.org>]

Hi,

I'm using HBase 0.94.5 with thrift server. I'm trying to get the rows from
HBase using
org.apache.hadoop.hbase.thrift.generated.Hbase.Client.getRows(ByteBuffer,
List<ByteBuffer>, Map<ByteBuffer, ByteBuffer>) but it is giving results
very slowly (around 2 mins for 100 rows). For larger number of records,
there is no response.

I've two region server and a total of 128 regions. Total data size is
around 250GB (250 million records) uniformly distributed across regions.

Regionserver only show the following in its log:

2013-04-11 20:11:47,882 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Stats: total=46.81 MB,
free=970.8 MB, max=1017.61 MB, blocks=591, accesses=33804, hits=2109,
hitRatio=6.23%, , cachingAccesses=2701, cachingHits=2109,
cachingHitsRatio=78.08%, , evictions=0, evicted=0, evictedPerRun=NaN
2013-04-11 20:12:11,236 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer:?
org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
multi(org.apache.hadoop.hbase.client.MultiAction@5e7c7894), rpc version=1,
client version=29, methodsFingerPrint=-1368823753 from
192.168.145.195:52313after 61519 ms, since caller disconnected at
org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576) at
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549) at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
2013-04-11 20:12:22,634 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer:?
org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
multi(org.apache.hadoop.hbase.client.MultiAction@5e7c7894), rpc version=1,
client version=29, methodsFingerPrint=-1368823753 from
192.168.145.195:52313after 72917 ms, since caller disconnected at
org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576) at
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549) at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)

Any idea what might be wrong here?

Thanks,
Anand