Indefinite pause while trying to cleanup data

[Jorge Figueira <jo...@wit-software.com>]

Hi,

 

I have reported  <https://issues.apache.org/jira/browse/HBASE-15573>
https://issues.apache.org/jira/browse/HBASE-15573 and you didn't try to find
the cause probably it's my fault, because from the start I didn't explain
with great detail all facts clearly. 

 

After 2 months with Hbase working fine RPC java client start retrieving null
data . Hbase server tells it's all OK with Region and Master but it can't
scan any data with the same configuration I have from 2 months ago that was
working perfectly until yesterday. 

 

With Hbase shell it's possible to use  scan 'JAMES_MAILBOXES', {COLUMNS =>
['D'],FILTER =>
"(SingleColumnValueFilter('D','user',=,'binary:241421421421',true,true)) AND
(SingleColumnValueFilter('D','name',=,'binary:INBOX',true,true))"} , same
filter with RPC client didn't work and retrieved null data (Exception :
DEBUG [RpcServer.reader=2,bindAddress=HBASE,port=16020] ipc.RpcServer:
RpcServer.listener,port=16020: DISCONNECTING client HBASE1:36997 because
read count=-1. Number of active connections: 2).

 

After many tries(repairs, updates) I decided to clean all data and my client
start working again (without any configuration changed). Something occurred
with Hbase and no one did any change configuration file inside Hbase (server
and client) or network. Clearly was an issue inside Hbase.

 

I have a snapshot of that state to try to understand what occurred and
prevent happening again.

 

 

With the bug described above when i try to export data from Hbase using
(hbase org.apache.hadoop.hbase.mapreduce.Export JAMES_MAILBOXES xpto.tmp)
some nasty exceptions occur
2016-03-31 15:37:39,745 DEBUG
[B.defaultRpcServer.handler=23,queue=2,port=16020] ipc.RpcServer:
B.defaultRpcServer.handler=23,queue=2,port=16020: callId: 36796 service:
ClientService methodName: Scan size: 586 connection: HBASE:57813
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region
JAMES_MAILBOXES,,1456359497674.b038389cb6f8d608abe86265dd90079f. is opening
on HBASE,16020,1459435051912
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HR
egionServer.java:2895)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.j
ava:947)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2
234)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.call
BlockingMethod(ClientProtos.java:32205)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)

 

 

 

Thanks,

Jorge Figueira   

WIT Software | Software Engineer 

  


 <http://goo.gl/UKnAoF> Call Me 

  

This email was sent under WIT Software's
<https://www.wit-software.com/confidentiality-policy/> Confidentiality
Policy