Re: Exception in HBase get

[mukund murrali <mu...@gmail.com>]

Hi

Few more pointers in the issue. Let's say there are two cells with column
keys as A and B for same row. These two cells are present in two different
files. Cell B is deleted recently. While scanning StoreFileScanner has read
B first then reads A. Though the lexographical sorting is preserved in
individual files why does the scanner reads deleted cell B first? This is
the issue causing problem in reads. Because of this, we have commented the
else part which throws IllegalStateException. Will this commenting cause
any issues(though no issues till now). Please someone enlighten more on
this issue please. We are using 1.2.5 stable version

On Mon, 10 Jul 2017 at 7:29 PM, mukund murrali <mu...@gmail.com>
wrote:

> +dev
>
> Regards,
>
> Mukund Murrali
>
> On Mon, Jul 10, 2017 at 1:17 PM, mukund murrali <mu...@gmail.com>
> wrote:
>
>> I know the row key. So in what way it will be helpful in analyzing this
>> issue?
>>
>> Regards,
>>
>> Mukund Murrali
>>
>> On Mon, Jul 10, 2017 at 5:32 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Can you find the hfile where this exception happens when doing get /
>>> scan ?
>>>
>>> Unfortunately the log didn't contain row key.
>>> Here is small change which would log row key:
>>>
>>> https://pastebin.com/XU5hCLXq
>>>
>>> Cheers
>>>
>>> On Thu, Jul 6, 2017 at 10:17 PM, Graceline Abigail Prem Kumar <
>>> pgabigail95@gmail.com> wrote:
>>>
>>> > Hi
>>> >
>>> > We are currently used Hbase 1.2.5. This exception occurs frequently,
>>> both
>>> > on the server and on the client side.
>>> >
>>> > Regards,
>>> > Graceline Abigail P
>>> >
>>> > On Thu, Jul 6, 2017 at 11:08 AM, Graceline Abigail Prem Kumar <
>>> > pgabigail95@gmail.com> wrote:
>>> >
>>> > > Hi
>>> > >
>>> > > We have got an IllegalStateException during HBase get operation. And
>>> > we're
>>> > > not sure about the cause. Here is the exception trace. What could the
>>> > > problem be?
>>> > >
>>> > > Caused by: java.lang.IllegalStateException: isDelete failed:
>>> > > deleteBuffer=22, qualifier=21, timestamp=1487055525513, comparison
>>> > result:
>>> > > 1
>>> > >     at
>>> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(
>>> > > ScanDeleteTracker.java:147)
>>> > >     at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.
>>> > > match(ScanQueryMatcher.java:395)
>>> > >     at org.apache.hadoop.hbase.regionserver.StoreScanner.
>>> > > next(StoreScanner.java:529)
>>> > >     at org.apache.hadoop.hbase.regionserver.KeyValueHeap.
>>> > > next(KeyValueHeap.java:150)
>>> > >     at
>>> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.
>>> > > populateResult(HRegion.java:5731)
>>> > >     at
>>> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.
>>> > > nextInternal(HRegion.java:5894)
>>> > >     at org.apache.hadoop.hbase.regionserver.HRegion$
>>> > > RegionScannerImpl.nextRaw(HRegion.java:5668)
>>> > >     at org.apache.hadoop.hbase.regionserver.HRegion$
>>> > > RegionScannerImpl.next(HRegion.java:5645)
>>> > >     at org.apache.hadoop.hbase.regionserver.HRegion$
>>> > > RegionScannerImpl.next(HRegion.java:5631)
>>> > >     at org.apache.hadoop.hbase.regionserver.HRegion.get(
>>> > > HRegion.java:6829)
>>> > >     at org.apache.hadoop.hbase.regionserver.HRegion.get(
>>> > > HRegion.java:6807)
>>> > >     at org.apache.hadoop.hbase.regionserver.RSRpcServices.
>>> > > get(RSRpcServices.java:2049)
>>> > >     at org.apache.hadoop.hbase.protobuf.generated.
>>> > >
>>> ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33644)
>>> > >     at
>>> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2188)
>>> > >     ... 4 more
>>> > >
>>> > > Mon Jul 03 04:34:07 PDT 2017, RpcRetryingCaller{
>>> > globalStartTime=1499081647183,
>>> > > pause=100, retries=35}, java.io.IOException: java.io.IOException:
>>> > isDelete
>>> > > failed: deleteBuffer=22, qualifier=21, timestamp=1487055525513,
>>> > comparison
>>> > > result: 1
>>> > >     at
>>> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2239)
>>> > >     at org.apache.hadoop.hbase.ipc.Ca
>>> llRunner.run(CallRunner.java:112)
>>> > >     at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(
>>> > > RpcExecutor.java:133)
>>> > >     at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.
>>> > > java:108)
>>> > >     at java.lang.Thread.run(Thread.java:745)
>>> > >
>>> > > Regards,
>>> > > Graceline Abigail P
>>> > >
>>> >
>>>
>>
>>
> --
Regards,

Mukund Murrali