You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jack Jiang (Jira)" <ji...@apache.org> on 2020/06/19 09:16:00 UTC
[jira] [Updated] (HBASE-16151) what's wrong when using bulkload to
load file to table in hbase ?
[ https://issues.apache.org/jira/browse/HBASE-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jack Jiang updated HBASE-16151:
-------------------------------
Description: (was: here is the code and logs below:
HTable table = new HTable(tempConf,tableName);
LoadIncrementalHFiles loader = new LoadIncrementalHFiles(conf);
loader.doBulkLoad(dir, table);
13:49:15,806 DEBUG ProtobufRpcEngine:253 - Call: getBlockLocations took 0ms
13:49:15,821 DEBUG DFSClient:273 - newInfo = LocatedBlocks{
fileLength=1051
underConstruction=false
blocks=[LocatedBlock
{BP-644339120-xxxx-1458531863647:blk_1075272092_1543930; getBlockSize()=1051; corrupt=false; offset=0; locs=[xxxx:50010, xxxx:50010, xxxx:50010]; storageIDs=[DS-d09ae035-279c-408c-8272-d84eb29f3e3b, DS-30754a89-526a-40ca-b0e0-f4400b0527bb, DS-856412f0-e865-43c0-bd60-4650b7ff275d]; storageTypes=[DISK, DISK, DISK]}
]
lastLocatedBlock=LocatedBlock
{BP-644339120-xxxx-1458531863647:blk_1075272092_1543930; getBlockSize()=1051; corrupt=false; offset=0; locs=[xxxx:50010, xxxx:50010, xxxx:50010]; storageIDs=[DS-d09ae035-279c-408c-8272-d84eb29f3e3b, DS-30754a89-526a-40ca-b0e0-f4400b0527bb, DS-856412f0-e865-43c0-bd60-4650b7ff275d]; storageTypes=[DISK, DISK, DISK]}
isLastBlockComplete=true}
13:49:15,821 DEBUG Client:1025 - IPC Client (1094238221) connection to /xxxx:8020 from uatxj990267 sending #4
13:49:15,821 DEBUG Client:1082 - IPC Client (1094238221) connection to /xxxx:8020 from uatxj990267 got value #4
13:49:15,821 DEBUG ProtobufRpcEngine:253 - Call: getFileInfo took 0ms
13:49:15,821 DEBUG DFSClient:961 - Connecting to datanode xxxx:50010
13:49:15,821 DEBUG Client:1025 - IPC Client (1094238221) connection to /xxxx:8020 from uatxj990267 sending #5
13:49:15,821 DEBUG Client:1082 - IPC Client (1094238221) connection to /xxxx:8020 from uatxj990267 got value #5
13:49:15,821 DEBUG ProtobufRpcEngine:253 - Call: getServerDefaults took 0ms
13:49:15,837 DEBUG SaslDataTransferClient:244 - SASL client skipping handshake in unsecured configuration for addr = /xxxx, datanodeId = xxxx:50010
13:49:15,899 DEBUG DFSClient:961 - Connecting to datanode xxxx:50010
13:49:15,915 INFO LoadIncrementalHFiles:517 - Trying to load hfile=hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68 first=14672656058391 last=14672656058393
13:49:15,946 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [
{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}
]
13:49:16,258 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 7,4 replyHeader:: 7,163216552663,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s
{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:16,258 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [\{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}]
13:49:16,773 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 8,4 replyHeader:: 8,163216552663,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s\{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:16,773 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [
{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}
]
13:49:17,788 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 9,4 replyHeader:: 9,163216552666,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s
{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:17,788 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [\{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}]
13:49:19,801 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 10,4 replyHeader:: 10,163216552668,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s\{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:19,801 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [
{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}
]
13:49:23,811 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 11,4 replyHeader:: 11,163216552675,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s
{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:23,811 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [\{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}]
13:49:25,824 DEBUG Client:1185 - IPC Client (1094238221) connection to /xxxx:8020 from uatxj990267: closed
13:49:25,824 DEBUG Client:980 - IPC Client (1094238221) connection to /xxxx:8020 from uatxj990267: stopped, remaining connections 0
13:49:33,874 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 12,4 replyHeader:: 12,163216552690,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s\{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:33,874 DEBUG ClientCnxn:717 - Got ping response for sessionid: 0x254ba19dd2d97eb after 1ms
13:49:33,874 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [
{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}
]
13:49:43,935 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 13,4 replyHeader:: 13,163216552705,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s
{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:43,935 DEBUG ClientCnxn:717 - Got ping response for sessionid: 0x254ba19dd2d97eb after 1ms
13:49:43,935 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [\{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}]
13:49:53,949 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 14,4 replyHeader:: 14,163216552723,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s\{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:49:53,949 DEBUG ClientCnxn:717 - Got ping response for sessionid: 0x254ba19dd2d97eb after 1ms
13:49:53,951 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [
{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}
]
13:50:03,994 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 15,4 replyHeader:: 15,163216552738,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s
{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:50:03,996 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [\{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}]
13:50:04,193 DEBUG ClientCnxn:717 - Got ping response for sessionid: 0x254ba19dd2d97eb after 200ms
13:50:17,326 DEBUG ClientCnxn:717 - Got ping response for sessionid: 0x254ba19dd2d97eb after 0ms
13:50:24,172 DEBUG ClientCnxn:818 - Reading reply sessionid:0x254ba19dd2d97eb, packet:: clientPath:null serverPath:null finished:false header:: 16,4 replyHeader:: 16,163216552771,0 request:: '/hbase-unsecure/table/emp,F response:: #ffffffff000146d61737465723a363030303017ffffff951415ffffff9f372d375042554680,s\{163215998528,163216219499,1467010385319,1467103321910,8,0,0,0,31,0,163215998528}
13:50:24,175 DEBUG LoadIncrementalHFiles:618 - Going to connect to server region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335 for row with hfile group [
{[B@281d9e15,hdfs://xxxx:8020/apps/hbase/data/info/info/32d34121d62b4327a8303ee55bb98b68}
]
13:50:24,182 INFO RpcRetryingCaller:129 - Call exception, tries=10, retries=35, started=68251 ms ago, cancelled=false, msg=row '' on table 'emp' at region=emp,,1467010385314.eb65b8443faee7c72ee1f003f1b8e7ba., hostname=xxxx002.xxxx,60020,1467195299232, seqNum=335)
> what's wrong when using bulkload to load file to table in hbase ?
> ------------------------------------------------------------------
>
> Key: HBASE-16151
> URL: https://issues.apache.org/jira/browse/HBASE-16151
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
> Reporter: Jack Jiang
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)