You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jean-Marc Spaggiari <je...@spaggiari.org> on 2019/04/11 15:38:46 UTC
Any bulkload issue with 2.2.0?
Trying to bulkload a single HFile in a single region empty table on a
sleeping 2.2.0 cluster, I get this:
2019-04-11 11:22:40,594 INFO [LoadIncrementalHFiles-0] compress.CodecPool:
Got brand-new decompressor [.snappy]
2019-04-11 11:22:40,632 INFO [LoadIncrementalHFiles-0]
tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23
first=Optional[\x02\x02\x01\x02\x03\x01\x02\x03\x03\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
last=Optional[\x02\x02\x02\x01\x08\x01\x05\x01\x00\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
2019-04-11 11:22:40,737 WARN [LoadIncrementalHFiles-1]
tool.LoadIncrementalHFiles: Attempt to bulk load region containing into
table stones5 with files [family:A path:hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23]
failed. This is recoverable and they will be retried.
2019-04-11 11:22:40,746 INFO [main] tool.LoadIncrementalHFiles: Split
occurred while grouping HFiles, retry attempt 1 with 1 files remaining to
group or split
2019-04-11 11:22:40,758 INFO [LoadIncrementalHFiles-2]
tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23
first=Optional[\x02\x02\x01\x02\x03\x01\x02\x03\x03\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
last=Optional[\x02\x02\x02\x01\x08\x01\x05\x01\x00\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
2019-04-11 11:22:40,816 WARN [LoadIncrementalHFiles-3]
tool.LoadIncrementalHFiles: Attempt to bulk load region containing into
table stones5 with files [family:A path:hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23]
failed. This is recoverable and they will be retried.
2019-04-11 11:22:40,824 INFO [main] tool.LoadIncrementalHFiles: Split
occurred while grouping HFiles, retry attempt 2 with 1 files remaining to
group or split
Bulkload seems to think that the destination table is splitting while
loading.
Destination table:
jmspaggi@node8:~/Othello$ hdfs dfs -ls /hbase/data/default/stones5/*
Found 1 items
-rw-r--r-- 3 hbase supergroup 507 2019-04-11 11:22
/hbase/data/default/stones5/.tabledesc/.tableinfo.0000000001
Found 2 items
-rw-r--r-- 3 hbase supergroup 42 2019-04-11 11:22
/hbase/data/default/stones5/052f1fdf28ae75754d28f2ed7fafd6c6/.regioninfo
drwxr-xr-x - hbase supergroup 0 2019-04-11 11:22
/hbase/data/default/stones5/052f1fdf28ae75754d28f2ed7fafd6c6/A
And meta seems to be clean:
stones5 column=table:state, timestamp=1554996138033, value=\x08\x00
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:regioninfo, timestamp=1554996137061, value={ENCODED =>
052f1fdf28ae75754d28f2ed7fafd6c6, NAME =>
'stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.', STARTKEY => '',
ENDKEY => ''}
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:seqnumDuringOpen, timestamp=1554996137061,
value=\x00\x00\x00\x00\x00\x00\x00\x02
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:server, timestamp=1554996137061, value=
node6.distparser.com:16020
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:serverstartcode, timestamp=1554996137061, value=1554893230076
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6. column=info:sn,
timestamp=1554996136579, value=node6.distparser.com,16020,1554893230076
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:state, timestamp=1554996137061, value=OPEN
Looking at the code, it uses some deprecated functions and types.
I went the dirty way and just manually moved all my files into the table
region, but I think there is something to be looked at here :-/
JMS