confirmed procedure for repairing hole in hbase metadata

[Geoff Hendrey <gh...@decarta.com>]

Stack,

Thanks again for providing us this procedure. We had 5 holes in a big multi-terabyte table. We were able to repair them all and get back to work. Regarding your question of what caused the holes in the first place: we have no idea, unfortunately. Our load is read, write, and update and the rows contain about 1 MB of data in 1 column.

-geoff

-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Thursday, August 18, 2011 11:29 AM
To: Rohit Nigam
Cc: Geoff Hendrey; user@hbase.apache.org; Search
Subject: Re: version mismatch exception

On Wed, Aug 17, 2011 at 1:12 PM, Rohit Nigam <rn...@decarta.com> wrote:
> Hi St.Ack
> The region in the file System are good, all I am looking is to change the end key of that region in the .META. table so that chaining problem goes away .The way I am planning to do is to get the HRegionInfo object for that existing region key from the .META. table . Create a new HRegionInfo obj with the updated endkey , start key and regionid being the same as from the result above and do a put in the .META. table. I think I just change the endkey and nothing else it will not create  a new row in .META. table and would just update the existing row. Please confirm if my theory is right.

1. Close the existing region if open.
2. Read the existing hregioninfo.
3. Use existing hregioninfo htabledescriptor and this construction,
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HRegionInfo.html#HRegionInfo(org.apache.hadoop.hbase.HTableDescriptor,
byte[], byte[]), to  make a new hregioninfo.   It will have a
different encoding to the original.
4. Insert the new entry in .META. using new hregioninfo name as row
key and add an info:regioninfo whose contents is the serialized new
hregioninfo.
5. Delete the original row from .META.
6. Assign the new region.

If you want the data from the old region in the new region, then you
should copy any files in that are under the old entries directory into
the new region directory (find the regions by using the encoded name;
the encoded name is an attribute of hregioninfo).  After copying in
the data, you'll need to reassign the region.  The files are only
noticed on region open.

St.Ack