You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "dong.yajun" <do...@gmail.com> on 2015/05/19 04:38:42 UTC
(info) How can i load data from CDH4.3.0 to CDH5.4.0 in Hbase
hello list,
is there a way to load the existing data(HFiles) from CDH4.3.0 to CDH5.4.0?
we use the complete bulkload utility which reference the link:
http://hbase.apache.org/0.94/book/ops_mgt.html#completebulkload
the command: hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles
/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/ IM_ItemPrice
the region of *096518a1aa5823c4aec9477d7b1b63cf *which contains several
family names: BaseInfo / of / ol / q4s etc.
but it seems not work, see the output after command typed below:
15/05/19 09:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load
hfile=hdfs://nameservice1/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/
*BaseInfo*/88fc1240fa8f4e31aa27469b7bd66750
first=e04116151155a5fae8dfa37281df89304c5e62219c31a761024cfac80f8e204c
last=ffffbaa9f23ae8fd718aa888f814665e19d04fcdee09de9926c8690db00af905
15/05/19 09:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load
hfile=hdfs://nameservice1/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/
*ol*/2a886066311343f98737ad2e4e804260
first=e045a94bb684ce8bbb5cf34b9e0dd939c03946bd445711204b30c17d72b55874
last=ffff7f5766b39d218e772dc357d9588b7a7363c03b3b7a07be7bcbb41dc267b3
15/05/19 09:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load
hfile=hdfs://nameservice1/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/
*of*/c01e895d483b4c86beb4eeae503e8fa9
first=e046037883152a81168507bf9faefc8ba716f15a6a028b81cbf83e2894896ec3
last=fff93051701920c2848f091369e484b685f42ed7a1378ffcb4e9dddf8bcd7ef7
15/05/19 10:09:17 INFO client.RpcRetryingCaller: Call exception, tries=10,
retries=35, started=677650 ms ago, cancelled=false, msg=row '' on table
'IM_ItemPrice' at
region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
hostname=ssspark03,60020,1431768671791, seqNum=2
15/05/19 10:10:32 INFO client.RpcRetryingCaller: Call exception, tries=11,
retries=35, started=752931 ms ago, cancelled=false, msg=row '' on table
'IM_ItemPrice' at
region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
hostname=ssspark03,60020,1431768671791, seqNum=2
15/05/19 10:11:48 INFO client.RpcRetryingCaller: Call exception, tries=12,
retries=35, started=828151 ms ago, cancelled=false, msg=row '' on table
'IM_ItemPrice' at
region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
hostname=ssspark03,60020,1431768671791, seqNum=2
15/05/19 10:13:03 INFO client.RpcRetryingCaller: Call exception, tries=13,
retries=35, started=903409 ms ago, cancelled=false, msg=row '' on table
'IM_ItemPrice' at
region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
hostname=ssspark03,60020,1431768671791, seqNum=2
15/05/19 10:14:18 INFO client.RpcRetryingCaller: Call exception, tries=14,
retries=35, started=978634 ms ago, cancelled=false, msg=row '' on table
'IM_ItemPrice' at
region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
hostname=ssspark03,60020,1431768671791, seqNum=2
15/05/19 10:15:33 INFO client.RpcRetryingCaller: Call exception, tries=15,
retries=35, started=1054003 ms ago, cancelled=false, msg=row '' on table
'IM_ItemPrice' at
region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
hostname=ssspark03,60020,1431768671791, seqNum=2
..
....
any suggestion?
--
*Ric Dong*
Re: (info) How can i load data from CDH4.3.0 to CDH5.4.0 in Hbase
Posted by Esteban Gutierrez <es...@cloudera.com>.
Hello Dong,
What do you see in the RS logs from ssspark03? bulkloading HFiles from 0.94
into HBase 1.0 should work without any major issue as long as all the store
files are using the HFileV2 format, howecer if you are trying to bulkload
HFileV1 files then you need to major compact the table from HBase 0.94 and
bulkload the newly created HFileV2 files.
thanks,
esteban.
--
Cloudera, Inc.
On Mon, May 18, 2015 at 7:38 PM, dong.yajun <do...@gmail.com> wrote:
> hello list,
>
> is there a way to load the existing data(HFiles) from CDH4.3.0 to CDH5.4.0?
>
> we use the complete bulkload utility which reference the link:
> http://hbase.apache.org/0.94/book/ops_mgt.html#completebulkload
>
> the command: hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles
> /tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/ IM_ItemPrice
>
> the region of *096518a1aa5823c4aec9477d7b1b63cf *which contains several
> family names: BaseInfo / of / ol / q4s etc.
>
> but it seems not work, see the output after command typed below:
>
> 15/05/19 09:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load
>
> hfile=hdfs://nameservice1/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/
> *BaseInfo*/88fc1240fa8f4e31aa27469b7bd66750
> first=e04116151155a5fae8dfa37281df89304c5e62219c31a761024cfac80f8e204c
> last=ffffbaa9f23ae8fd718aa888f814665e19d04fcdee09de9926c8690db00af905
> 15/05/19 09:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load
>
> hfile=hdfs://nameservice1/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/
> *ol*/2a886066311343f98737ad2e4e804260
> first=e045a94bb684ce8bbb5cf34b9e0dd939c03946bd445711204b30c17d72b55874
> last=ffff7f5766b39d218e772dc357d9588b7a7363c03b3b7a07be7bcbb41dc267b3
> 15/05/19 09:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load
>
> hfile=hdfs://nameservice1/tmp/IM_ItemPrice/096518a1aa5823c4aec9477d7b1b63cf/
> *of*/c01e895d483b4c86beb4eeae503e8fa9
> first=e046037883152a81168507bf9faefc8ba716f15a6a028b81cbf83e2894896ec3
> last=fff93051701920c2848f091369e484b685f42ed7a1378ffcb4e9dddf8bcd7ef7
>
> 15/05/19 10:09:17 INFO client.RpcRetryingCaller: Call exception, tries=10,
> retries=35, started=677650 ms ago, cancelled=false, msg=row '' on table
> 'IM_ItemPrice' at
> region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
> hostname=ssspark03,60020,1431768671791, seqNum=2
> 15/05/19 10:10:32 INFO client.RpcRetryingCaller: Call exception, tries=11,
> retries=35, started=752931 ms ago, cancelled=false, msg=row '' on table
> 'IM_ItemPrice' at
> region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
> hostname=ssspark03,60020,1431768671791, seqNum=2
> 15/05/19 10:11:48 INFO client.RpcRetryingCaller: Call exception, tries=12,
> retries=35, started=828151 ms ago, cancelled=false, msg=row '' on table
> 'IM_ItemPrice' at
> region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
> hostname=ssspark03,60020,1431768671791, seqNum=2
> 15/05/19 10:13:03 INFO client.RpcRetryingCaller: Call exception, tries=13,
> retries=35, started=903409 ms ago, cancelled=false, msg=row '' on table
> 'IM_ItemPrice' at
> region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
> hostname=ssspark03,60020,1431768671791, seqNum=2
> 15/05/19 10:14:18 INFO client.RpcRetryingCaller: Call exception, tries=14,
> retries=35, started=978634 ms ago, cancelled=false, msg=row '' on table
> 'IM_ItemPrice' at
> region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
> hostname=ssspark03,60020,1431768671791, seqNum=2
> 15/05/19 10:15:33 INFO client.RpcRetryingCaller: Call exception, tries=15,
> retries=35, started=1054003 ms ago, cancelled=false, msg=row '' on table
> 'IM_ItemPrice' at
> region=IM_ItemPrice,,1432000677144.36c13c3160de2e67e4fdb1d77c3c9ade.,
> hostname=ssspark03,60020,1431768671791, seqNum=2
> ..
> ....
>
> any suggestion?
>
> --
> *Ric Dong*
>