You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jean-Marc Spaggiari <je...@spaggiari.org> on 2019/04/11 15:38:46 UTC

Any bulkload issue with 2.2.0?

Trying to bulkload a single HFile in a single region empty table on a
sleeping 2.2.0 cluster, I get this:

2019-04-11 11:22:40,594 INFO  [LoadIncrementalHFiles-0] compress.CodecPool:
Got brand-new decompressor [.snappy]
2019-04-11 11:22:40,632 INFO  [LoadIncrementalHFiles-0]
tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23
first=Optional[\x02\x02\x01\x02\x03\x01\x02\x03\x03\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
last=Optional[\x02\x02\x02\x01\x08\x01\x05\x01\x00\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
2019-04-11 11:22:40,737 WARN  [LoadIncrementalHFiles-1]
tool.LoadIncrementalHFiles: Attempt to bulk load region containing  into
table stones5 with files [family:A path:hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23]
failed.  This is recoverable and they will be retried.
2019-04-11 11:22:40,746 INFO  [main] tool.LoadIncrementalHFiles: Split
occurred while grouping HFiles, retry attempt 1 with 1 files remaining to
group or split
2019-04-11 11:22:40,758 INFO  [LoadIncrementalHFiles-2]
tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23
first=Optional[\x02\x02\x01\x02\x03\x01\x02\x03\x03\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
last=Optional[\x02\x02\x02\x01\x08\x01\x05\x01\x00\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
2019-04-11 11:22:40,816 WARN  [LoadIncrementalHFiles-3]
tool.LoadIncrementalHFiles: Attempt to bulk load region containing  into
table stones5 with files [family:A path:hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23]
failed.  This is recoverable and they will be retried.
2019-04-11 11:22:40,824 INFO  [main] tool.LoadIncrementalHFiles: Split
occurred while grouping HFiles, retry attempt 2 with 1 files remaining to
group or split

Bulkload seems to think that the destination table is splitting while
loading.

Destination table:
 jmspaggi@node8:~/Othello$ hdfs dfs -ls /hbase/data/default/stones5/*
Found 1 items
-rw-r--r--   3 hbase supergroup        507 2019-04-11 11:22
/hbase/data/default/stones5/.tabledesc/.tableinfo.0000000001
Found 2 items
-rw-r--r--   3 hbase supergroup         42 2019-04-11 11:22
/hbase/data/default/stones5/052f1fdf28ae75754d28f2ed7fafd6c6/.regioninfo
drwxr-xr-x   - hbase supergroup          0 2019-04-11 11:22
/hbase/data/default/stones5/052f1fdf28ae75754d28f2ed7fafd6c6/A


And meta seems to be clean:
 stones5 column=table:state, timestamp=1554996138033, value=\x08\x00
 stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:regioninfo, timestamp=1554996137061, value={ENCODED =>
052f1fdf28ae75754d28f2ed7fafd6c6, NAME =>
'stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.', STARTKEY => '',
ENDKEY => ''}
 stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:seqnumDuringOpen, timestamp=1554996137061,
value=\x00\x00\x00\x00\x00\x00\x00\x02
 stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:server, timestamp=1554996137061, value=
node6.distparser.com:16020
 stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:serverstartcode, timestamp=1554996137061, value=1554893230076
 stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6. column=info:sn,
timestamp=1554996136579, value=node6.distparser.com,16020,1554893230076
 stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:state, timestamp=1554996137061, value=OPEN

Looking at the code, it uses some deprecated functions and types.

I went the dirty way and just manually moved all my files into the table
region, but I think there is something to be looked at here :-/

JMS