You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2017/11/24 11:06:40 UTC
Re: java.lang.IllegalStateException: Invalid currKeyLen 33554496
What if you add '-m' to the HFile command ?
Any encoding you use for the underlying table ?
Did you use table API to write data or thru bulk load ?
Cheers
On Thu, Nov 23, 2017 at 10:37 PM, Weizhan Zeng <qg...@gmail.com> wrote:
>
> Hi，guys
>
> I user hbase-1.2.6 , and I found a strange problem， the key is very short
> but get “ Invalid currKeyLen 33554496 "
>
> Is anyone had met that ?
>
>
> hbase(main):018:0> get 'test', '20#1960620#20171026'
> COLUMN CELL
>
> ERROR: java.io.IOException: java.lang.IllegalStateException: Invalid
> currKeyLen 33554496 or currValueLen 706. Block offset: 3773220580287066162,
> block length: 158499, position: 0 (without header).
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.
> handleException(HRegion.java:5600)
> at org.apache.hadoop.hbase.regionserver.HRegion$
> RegionScannerImpl.<init>(HRegion.java:5570)
> at org.apache.hadoop.hbase.regionserver.HRegion.
> instantiateRegionScanner(HRegion.java:2574)
> at org.apache.hadoop.hbase.regionserver.HRegion.
> getScanner(HRegion.java:2560)
> at org.apache.hadoop.hbase.regionserver.HRegion.
> getScanner(HRegion.java:2541)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(
> HRegion.java:6830)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(
> HRegion.java:6809)
> at org.apache.hadoop.hbase.regionserver.RSRpcServices.
> get(RSRpcServices.java:2049)
> at org.apache.hadoop.hbase.protobuf.generated.
> ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33644)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(
> RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.
> java:108)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: Invalid currKeyLen 33554496
> or currValueLen 706. Block offset: 3773220580287066162, block length:
> 158499, position: 0 (without header).
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.
> checkKeyValueLen(HFileReaderV2.java:985)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.
> readKeyValueLen(HFileReaderV3.java:245)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.
> updateCurrBlock(HFileReaderV2.java:962)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.
> loadBlockAndSeekToKey(HFileReaderV2.java:933)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$
> AbstractScannerV2.seekTo(HFileReaderV2.java:655)
> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$
> AbstractScannerV2.seekTo(HFileReaderV2.java:599)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
> seekAtOrAfter(StoreFileScanner.java:268)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
> seek(StoreFileScanner.java:173)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.
> seekScanners(StoreScanner.java:350)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.<
> init>(StoreScanner.java:199)
> at org.apache.hadoop.hbase.regionserver.HStore.
> getScanner(HStore.java:2077)
> at org.apache.hadoop.hbase.regionserver.HRegion$
> RegionScannerImpl.<init>(HRegion.java:5556)
> ... 12 more
>
>
>
> hbase org.apache.hadoop.hbase.io.hfile.HFile -p -f
> /hbase/data/default/test/5e71e0ac15c82619de9602b713aa8cb9/f/
> b6695b40cdb14c189c7cc41fa2dd21e0 -w '20#1960620#20171026'
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/home/hadoop/hbase/
> lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/home/hadoop/hadoop/
> share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 2017-11-24 14:31:25,756 INFO [main] hfile.CacheConfig: Created
> cacheConfig: CacheConfig:disabled
> K: 20#1960620#20171026/f:d/1511428582215/Put/vlen=68/seqid=80546 V:
> ["20#1960620#20171026#20171026#1", "20#1960620#20171026#20171026#2"]
> Scanned kv count -> 1
Re: java.lang.IllegalStateException: Invalid currKeyLen 33554496
Posted by Weizhan Zeng <qg...@gmail.com>.
Hi， Ted
After add -m , I got this
And I use snappy in HDFS , my table DATA_BLOCK_ENCODING is NONE.
Also I used python write data by thrift server and I readed by hbase shell and python api。
> 在 2017年11月24日，下午7:06，Ted Yu <yu...@gmail.com> 写道：
>
> What if you add '-m' to the HFile command ?
>
> Any encoding you use for the underlying table ?
>
> Did you use table API to write data or thru bulk load ?
>
> Cheers
>
> On Thu, Nov 23, 2017 at 10:37 PM, Weizhan Zeng <qg...@gmail.com> wrote:
>
>>
>> Hi，guys
>>
>> I user hbase-1.2.6 , and I found a strange problem， the key is very short
>> but get “ Invalid currKeyLen 33554496 "
>>
>> Is anyone had met that ?
>>
>>
>> hbase(main):018:0> get 'test', '20#1960620#20171026'
>> COLUMN CELL
>>
>> ERROR: java.io.IOException: java.lang.IllegalStateException: Invalid
>> currKeyLen 33554496 or currValueLen 706. Block offset: 3773220580287066162,
>> block length: 158499, position: 0 (without header).
>> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.
>> handleException(HRegion.java:5600)
>> at org.apache.hadoop.hbase.regionserver.HRegion$
>> RegionScannerImpl.<init>(HRegion.java:5570)
>> at org.apache.hadoop.hbase.regionserver.HRegion.
>> instantiateRegionScanner(HRegion.java:2574)
>> at org.apache.hadoop.hbase.regionserver.HRegion.
>> getScanner(HRegion.java:2560)
>> at org.apache.hadoop.hbase.regionserver.HRegion.
>> getScanner(HRegion.java:2541)
>> at org.apache.hadoop.hbase.regionserver.HRegion.get(
>> HRegion.java:6830)
>> at org.apache.hadoop.hbase.regionserver.HRegion.get(
>> HRegion.java:6809)
>> at org.apache.hadoop.hbase.regionserver.RSRpcServices.
>> get(RSRpcServices.java:2049)
>> at org.apache.hadoop.hbase.protobuf.generated.
>> ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33644)
>> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
>> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
>> at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(
>> RpcExecutor.java:133)
>> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.
>> java:108)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: java.lang.IllegalStateException: Invalid currKeyLen 33554496
>> or currValueLen 706. Block offset: 3773220580287066162, block length:
>> 158499, position: 0 (without header).
>> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.
>> checkKeyValueLen(HFileReaderV2.java:985)
>> at org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.
>> readKeyValueLen(HFileReaderV3.java:245)
>> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.
>> updateCurrBlock(HFileReaderV2.java:962)
>> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.
>> loadBlockAndSeekToKey(HFileReaderV2.java:933)
>> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$
>> AbstractScannerV2.seekTo(HFileReaderV2.java:655)
>> at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$
>> AbstractScannerV2.seekTo(HFileReaderV2.java:599)
>> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
>> seekAtOrAfter(StoreFileScanner.java:268)
>> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.
>> seek(StoreFileScanner.java:173)
>> at org.apache.hadoop.hbase.regionserver.StoreScanner.
>> seekScanners(StoreScanner.java:350)
>> at org.apache.hadoop.hbase.regionserver.StoreScanner.<
>> init>(StoreScanner.java:199)
>> at org.apache.hadoop.hbase.regionserver.HStore.
>> getScanner(HStore.java:2077)
>> at org.apache.hadoop.hbase.regionserver.HRegion$
>> RegionScannerImpl.<init>(HRegion.java:5556)
>> ... 12 more
>>
>>
>>
>> hbase org.apache.hadoop.hbase.io.hfile.HFile -p -f
>> /hbase/data/default/test/5e71e0ac15c82619de9602b713aa8cb9/f/
>> b6695b40cdb14c189c7cc41fa2dd21e0 -w '20#1960620#20171026'
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in [jar:file:/home/hadoop/hbase/
>> lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in [jar:file:/home/hadoop/hadoop/
>> share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/
>> impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> 2017-11-24 14:31:25,756 INFO [main] hfile.CacheConfig: Created
>> cacheConfig: CacheConfig:disabled
>> K: 20#1960620#20171026/f:d/1511428582215/Put/vlen=68/seqid=80546 V:
>> ["20#1960620#20171026#20171026#1", "20#1960620#20171026#20171026#2"]
>> Scanned kv count -> 1