You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Sharma, Avani" <ag...@ebay.com> on 2010/09/08 02:04:20 UTC

need to recover tables after restarting Hbase

I restarted hbase and hdfs today. When hbase came back, it did not show any tables.
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.6, r965666, Mon Jul 19 15:48:07 PDT 2010

hbase(main):002:0> list
0 row(s) in 6.1130 seconds

I recovered one table using add_table.rb.

hbase(main):005:0> list
TABLE1
1 row(s) in 6.0530 seconds


I have many more tables - probably 10 more. I don't remember their names. How do I recover them ?

A scan of .META to find regions shows as follows:
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.6, r965666, Mon Jul 19 15:48:07 PDT 2010
scan '.META.'
ROW                                                                          COLUMN+CELL

 TABLE1,,1282610231669                                                  column=info:server, timestamp=1283902008118, value=IP1:60020

 TABLE1,,1282610231669                                                  column=info:serverstartcode, timestamp=1283902008118, value=1283901768814

 TABLE2,,1282609302542                                                        column=info:server, timestamp=1283902008127, value=IP1:60020

 TABLE2,,1282609302542                                                        column=info:serverstartcode, timestamp=1283902008127, value=1283901768814

 TABLE3,,1282610731451                                                      column=info:server, timestamp=1283902010156, value=IP1:60020

 TABLE3,,1282610731451                                                      column=info:serverstartcode, timestamp=1283902010156, value=1283901768814

 TABLE3,KOENIG-OT.AT,1282610731451                                          column=info:server, timestamp=1283902010164, value=IP1:60020

 TABLE3,KOENIG-OT.AT,1282610731451                              column=info:serverstartcode, timestamp=1283902010164, value=1283901768814


 TABLE4,109739358,1282862857206                                 column=info:server, timestamp=1283902016305, value=IP1:60020

 TABLE4,109739358,1282862857206                                 column=info:serverstartcode, timestamp=1283902016305, value=1283901768814

 TABLE4,125574503,1282862857206                                 column=info:server, timestamp=1283902017325, value=IP1:60020

...
...


I could reproduce this issue by stopping and starting hbase again with the proper stop/start-hbase scripts.
I again see
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.6, r965666, Mon Jul 19 15:48:07 PDT 2010
hbase(main):001:0> list
0 row(s) in 0.1060 seconds
hbase(main):002:0>

Also in the master log, I notice -
2010-09-07 16:23:47,656 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current assignment of TABLE1,,1282610231669 is not valid;  serverAddress=IP1:60020, startCode=1283901768814
unknown.
2010-09-07 16:23:47,798 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current assignment of TABLE2,,1282609302542 is not valid;  serverAddress=IP1:60020, startCode=1283901768814 unknown.
2010-09-07 16:23:48,371 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current assignment of TABLE3,,1282610731451 is not valid;  serverAddress=IP2:60020, startCode=1283901768772 unknown.
2010-09-07 16:23:48,553 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current assignment of TABLE3,KOENIG-OT.AT,1282610731451 is not valid;  serverAddress=IP2:60020, startCode=128390
1768772 unknown.


Would you know what is going on here? Why is Hbase losing data and how can I recover all of it ?

Any help is appreciated.

Thanks,
-Avani