Storm-HBase: OperationConflictException

[王 纯超 <wa...@outlook.com>]

Hi all,

Below error occurs occasionally. What's the possible cause? I am using Storm 1.1.0. And I have two bolt writing to HBase, both have two tasks. But the other bolt  does not have this error.  Their writing speeds are approximately the same. And the keyspace is far more larger than the other. Its keys are more than likely to duplicate in one batch write.

Time    Error Host      Error Port      Error
2017-09-21T08:25:54.000Z        hadoop182       6702<http://hadoop182:8000/log?file=DPITopology-45-1505979705%2F6702%2Fworker.log>      org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 376 actions: org.apache.hadoop.hbase.exceptions.OperationConflictException: The operation with nonce {-6101012730784879159, 8964218331247407092} on row [c�Q�(E2�] may have already completed at org.apache.hadoop.hbase.regionserver.HRegionServer.startNonceOperation(HRegionServer.java:4351) at org.apache.hadoop.hbase.regionserver.HRegionServer.increment(HRegionServer.java:4315) at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3563) at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3496) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29931) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:116) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:96) at java.lang.Thread.run(Thread.java:745) : 1 time, org.apache.hadoop.hbase.exceptions.OperationConflictException: The operation with nonce {-6101012730784879159, 2091417366752441428} on row [c�Q�" x] may have already completed at org.apache.hadoop.hbase.regionserver.HRegionServer.startNonceOperation(HRegionServer.java:4351) at org.apache.hadoop.hbase.regionserver.HRegionServer.increment(HRegionServer.java:4315) at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3563) at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3496) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29931) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:116) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:96) at java.lang.Thread.run(Thread.java:745) : 1 time, org.apache.hadoop.hbase.exceptions.OperationConflictException: The operation with nonce {-6101012730784879159, 4457764140163160597} on row [c�Qگ�^�] may have already completed at org.apache.hadoop.hbase.regionserver.HRegionServer.startNonceOperation(HRegionServer.java:4351) at org.apache.hadoop.hbase.regionserver.HRegionServer.increment(HRegionServer.java:4315) at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3563) at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3496) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29931) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:116) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:96) at java.lang.Thread.run(Thread.java:745) : 1 time, org.apache.hadoop.hbase.exceptions.OperationConflictException: The operation with nonce {-6101012730784879159, -9075865740596317521} on row [c�Q�$
��] may have already completed at org.apache.hadoop.hbase.regionserver.HRegionServer.startNonceOperation(HRegionServer.java:4351) at org.apache.hadoop.hbase.regionserver.HRegionServer.increment(HRegionServer.java:4315) at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3563) at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3496) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29931) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:116) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:96) at java.lang.Thread.run(Thread.java:745) : 1 time, org.apache.hadoop.hbase.exceptions.OperationConflictException: The operation with nonce {-6101012730784879159, -6353626281138241034} on row [c�Q�fu(�] may have already completed at org.apache.hadoop.hbase.regionserver.HRegionServer.startNonceOperation(HRegionServer.java:4351) at org.apache.hadoop.hbase.regionserver.HRegionServer.increment(HRegionServer.java:4315) at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3563) at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3496) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29931) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:116) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:96) at java.lang.Thread.run(Thread.java:745) : 1 time, org.apache.hadoop.hbase.exceptions.OperationConflictException: The operation with nonce {-6101012730784879159, -752554279863447509} on row [c�Q�'լ] may have already completed at

________________________________
wangchunchao@outlook.com