You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Robert Gonzalez <Ro...@maxpointinteractive.com> on 2011/06/01 00:20:03 UTC
RE: wrong region exception
The script ran without the previous problem, but it did not fix the problem. When I ran hbck or check_meta.rb again they indicated that the problem was still there. Do I need to do something else in preparation before running check_meta?
Thanks,
Robert
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, May 31, 2011 2:57 PM
To: user@hbase.apache.org
Subject: Re: wrong region exception
Try adding this change:
Index: bin/check_meta.rb
===================================================================
--- bin/check_meta.rb (revision 1129468)
+++ bin/check_meta.rb (working copy)
@@ -127,11 +127,13 @@
scan = Scan.new()
scanner = metatable.getScanner(scan)
oldHRI = nil
-bad = nil
+bad = 0
while (result = scanner.next())
rowid = Bytes.toString(result.getRow())
rowidStr = java.lang.String.new(rowid)
bytes = result.getValue(HConstants::CATALOG_FAMILY,
HConstants::REGIONINFO_QUALIFIER)
+ next if not bytes
+ next if bytes.length == 0
hri = Writables.getHRegionInfo(bytes)
if oldHRI
if oldHRI.isOffline() && Bytes.equals(oldHRI.getStartKey(),
hri.getStartKey())
You might print out the result you have if the qualifier is null just to see what are the rows missing an HRegionInfo.
St.Ack
On Tue, May 31, 2011 at 9:02 AM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
> I'm trying my "nuclear" option: basically copy the data from the old db to a new one, skipping over bad regions. The bad news is that it is taking forever.
>
>
> I get a stack trace just trying to run check_meta.rb:
>
> maxpoint@c1-m02:/usr/lib/hbase/bin$ ./hbase org.jruby.Main
> check_meta.rb Writables.java:75:in
> `org.apache.hadoop.hbase.util.Writables.getWritable':
> java.lang.NullPointerException: null (NativeException)
> from Writables.java:119:in `org.apache.hadoop.hbase.util.Writables.getHRegionInfo'
> from NativeMethodAccessorImpl.java:-2:in `sun.reflect.NativeMethodAccessorImpl.invoke0'
> from NativeMethodAccessorImpl.java:39:in `sun.reflect.NativeMethodAccessorImpl.invoke'
> from DelegatingMethodAccessorImpl.java:25:in `sun.reflect.DelegatingMethodAccessorImpl.invoke'
> from Method.java:597:in `java.lang.reflect.Method.invoke'
> from JavaMethod.java:196:in `org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling'
> from JavaMethod.java:182:in `org.jruby.javasupport.JavaMethod.invoke_static'
> from JavaClass.java:371:in `org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute'
> ... 17 levels...
> from Main.java:183:in `org.jruby.Main.runInterpreter'
> from Main.java:120:in `org.jruby.Main.run'
> from Main.java:95:in `org.jruby.Main.main'
> Complete Java stackTrace
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75)
> at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:1
> 19)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> ava:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess
> orImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling(JavaMetho
> d.java:196)
> at
> org.jruby.javasupport.JavaMethod.invoke_static(JavaMethod.java:182)
> at
> org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute(JavaClass.
> java:371)
> at
> org.jruby.internal.runtime.methods.SimpleCallbackMethod.call(SimpleCal
> lbackMethod.java:81)
> at
> org.jruby.evaluator.EvaluationState.callNode(EvaluationState.java:571)
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 207)
> at
> org.jruby.evaluator.EvaluationState.localAsgnNode(EvaluationState.java
> :1254)
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 286)
> at
> org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533
> )
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 201)
> at
> org.jruby.evaluator.EvaluationState.whileNode(EvaluationState.java:179
> 3)
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 387)
> at
> org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533
> )
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 201)
> at
> org.jruby.evaluator.EvaluationState.rootNode(EvaluationState.java:1628
> )
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 356)
> at
> org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:164)
> at org.jruby.Ruby.eval(Ruby.java:278)
> at org.jruby.Ruby.compileOrFallbackAndRun(Ruby.java:306)
> at org.jruby.Main.runInterpreter(Main.java:238)
> at org.jruby.Main.runInterpreter(Main.java:183)
> at org.jruby.Main.run(Main.java:120)
> at org.jruby.Main.main(Main.java:95)
>
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Friday, May 27, 2011 12:43 AM
> To: user@hbase.apache.org
> Subject: Re: wrong region exception
>
> Robert:
>
> Looks like script already exists. Check bin/check_meta.rb. If you pass it --fix it should plug the hole. Read the head of the script for how to run it.
>
> Good luck,
> St.Ack
>
> On Thu, May 26, 2011 at 1:06 PM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
>> I sent the meta.txt to your saint.ack@gmail .com account due to the attachment.
>>
>> -----Original Message-----
>> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
>> Stack
>> Sent: Thursday, May 26, 2011 1:35 PM
>> To: user@hbase.apache.org
>> Subject: Re: wrong region exception
>>
>> On Thu, May 26, 2011 at 8:06 AM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
>>> It looks like an entire region is missing, here is the online table:
>>>
>>> urlhashv2,7FF1A5BF839C37078083B4F8267008F6,1303028235302.b0d55566fd0
>>> e
>>> 0 2ae98541618396aa7b1.
>>> c1-s03.atxd.maxpointinteractive.com:60030
>>> 7FF1A5BF839C37078083B4F8267008F6
>>> 80116D7E506D87ED39EAFFE784B5B590
>>> urlhashv2,8031483E0B3B7F587020FCBB764272D9,1305226123483.3ed065ad87f
>>> 8
>>> 9 aece6b994dd31b42b2a.
>>> c1-s33.atxd.maxpointinteractive.com:60030
>>> 8031483E0B3B7F587020FCBB764272D9
>>> 8041346D0B05617FA4B9152BFE9B18B9
>>>
>>> One ends at 80116D7E506D87ED39EAFFE784B5B590, but the next one doesn't start there.
>>>
>>
>> So, make sure you actually have a hole. Dump out your meta table:
>>
>> echo "scan '.META.'"| ./bin/hbase shell &> /tmp/meta.txt
>>
>> Then look ensure that there is a hole between the above regions (compare start and end keys... the end key of one region needs to match the start key of the next).
>>
>> If indeed a hole, you need to do a little surgery inserting a new missing region (hbck should fix this but it doesn't have the smarts just yet).
>>
>> Basically, you create a new region with start and end keys to fill the hole then you insert it into .META. and then assign it. There are some scripts in our bin directory that do various parts of this. I'm pretty sure its beyond any but a few figuring this mess out so if you do the above foot work and provide a few more details, I'll hack up something for you (and hopefully something generalized to be use by others later, and later to be integrated into hbck).
>>
>> St.Ack
>>
>
RE: wrong region exception
Posted by Robert Gonzalez <Ro...@maxpointinteractive.com>.
If I write manually, I get this:
hbase(main):005:0> put 'urlhashv4', "7BB16418308C2CB6B8AE56982781A5C7", 'url', "bogus"
ERROR: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region urlhashv4,7BB16418308C2CB6B8AE56982781A5C6,1308776365880.870c76fc43287036d776cca6c4ac6e6f.
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Wednesday, June 22, 2011 11:42 PM
To: user@hbase.apache.org
Subject: Re: wrong region exception
On Wed, Jun 22, 2011 at 2:23 PM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
> Stack,
>
> This script does work from one perspective. It makes an entry in the .meta. table with the missing region. But it does not create a region file for it. How does one go about doing that?
>
That will be created as soon as you've written enough edits to cause a flush.
St.Ack
RE: wrong region exception
Posted by Robert Gonzalez <Ro...@maxpointinteractive.com>.
Its been writing for a while now. When I try to read from that range, I get an exception that is caught. Is this normal?
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Wednesday, June 22, 2011 11:42 PM
To: user@hbase.apache.org
Subject: Re: wrong region exception
On Wed, Jun 22, 2011 at 2:23 PM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
> Stack,
>
> This script does work from one perspective. It makes an entry in the .meta. table with the missing region. But it does not create a region file for it. How does one go about doing that?
>
That will be created as soon as you've written enough edits to cause a flush.
St.Ack
Re: wrong region exception
Posted by Stack <st...@duboce.net>.
On Wed, Jun 22, 2011 at 2:23 PM, Robert Gonzalez
<Ro...@maxpointinteractive.com> wrote:
> Stack,
>
> This script does work from one perspective. It makes an entry in the .meta. table with the missing region. But it does not create a region file for it. How does one go about doing that?
>
That will be created as soon as you've written enough edits to cause a flush.
St.Ack
RE: wrong region exception
Posted by Robert Gonzalez <Ro...@maxpointinteractive.com>.
Stack,
This script does work from one perspective. It makes an entry in the .meta. table with the missing region. But it does not create a region file for it. How does one go about doing that?
Thanks,
Robert
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, May 31, 2011 6:39 PM
To: user@hbase.apache.org
Subject: Re: wrong region exception
On Tue, May 31, 2011 at 3:34 PM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
> The script doesn't work because it attempts to fix the hole by finding a region in the hdfs filesystem that fills the hole. But in this case there is no such file. The hole is just there.
>
OK. The fixup method has the left and right edges. Could use previous rows regioninfo and adjust it to fill the hole (or create a new one)? Something like the below?
diff --git a/bin/check_meta.rb b/bin/check_meta.rb index d874922..82c6ac0 100644
--- a/bin/check_meta.rb
+++ b/bin/check_meta.rb
@@ -80,32 +80,12 @@ def getConfiguration end
def fixup(leftEdge, rightEdge, metatable, fs, rootdir)
- plugged = nil
- # Try and fix the passed holes in meta.
- tabledir = HTableDescriptor::getTableDir(rootdir,
leftEdge.getTableDesc().getName())
- statuses = fs.listStatus(tabledir)
- for status in statuses
- next unless status.isDir()
- next if status.getPath().getName() == "compaction.dir"
- regioninfofile = Path.new(status.getPath(), ".regioninfo")
- unless fs.exists(regioninfofile)
- LOG.warn("Missing .regioninfo: " + regioninfofile.toString())
- next
- end
- is = fs.open(regioninfofile)
- hri = HRegionInfo.new()
- hri.readFields(is)
- is.close()
- next unless Bytes.equals(leftEdge.getEndKey(), hri.getStartKey())
- # TODO: Check against right edge to make sure this addition does
not overflow right edge.
- # TODO: Check that the schema matches both left and right edges schemas.
- p = Put.new(hri.getRegionName())
- p.add(HConstants::CATALOG_FAMILY,
HConstants::REGIONINFO_QUALIFIER, Writables.getBytes(hri))
- metatable.put(p)
- LOG.info("Plugged hole in .META. at: " + hri.toString())
- plugged = true
- end
- return plugged
+ hri = HRegionInfo.new(leftEdge.getTableDesc(),
leftEdge.getEndRow(), rightEdge.getStartRow())
+ p = Put.new(hri.getRegionName())
+ p.add(HConstants::CATALOG_FAMILY, HConstants::REGIONINFO_QUALIFIER,
Writables.getBytes(hri))
+ metatable.put(p)
+ LOG.info("Plugged hole in .META. at: " + hri.toString()) return true
end
fixup = isFixup()
St.Ack
RE: wrong region exception
Posted by Robert Gonzalez <Ro...@maxpointinteractive.com>.
I'll create a new script, fix_hole, that does this. We are going to let the copy finish first, then we will try this with the broken original table.
Thanks Stack,
Robert
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, May 31, 2011 6:39 PM
To: user@hbase.apache.org
Subject: Re: wrong region exception
On Tue, May 31, 2011 at 3:34 PM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
> The script doesn't work because it attempts to fix the hole by finding a region in the hdfs filesystem that fills the hole. But in this case there is no such file. The hole is just there.
>
OK. The fixup method has the left and right edges. Could use previous rows regioninfo and adjust it to fill the hole (or create a new one)? Something like the below?
diff --git a/bin/check_meta.rb b/bin/check_meta.rb index d874922..82c6ac0 100644
--- a/bin/check_meta.rb
+++ b/bin/check_meta.rb
@@ -80,32 +80,12 @@ def getConfiguration end
def fixup(leftEdge, rightEdge, metatable, fs, rootdir)
- plugged = nil
- # Try and fix the passed holes in meta.
- tabledir = HTableDescriptor::getTableDir(rootdir,
leftEdge.getTableDesc().getName())
- statuses = fs.listStatus(tabledir)
- for status in statuses
- next unless status.isDir()
- next if status.getPath().getName() == "compaction.dir"
- regioninfofile = Path.new(status.getPath(), ".regioninfo")
- unless fs.exists(regioninfofile)
- LOG.warn("Missing .regioninfo: " + regioninfofile.toString())
- next
- end
- is = fs.open(regioninfofile)
- hri = HRegionInfo.new()
- hri.readFields(is)
- is.close()
- next unless Bytes.equals(leftEdge.getEndKey(), hri.getStartKey())
- # TODO: Check against right edge to make sure this addition does
not overflow right edge.
- # TODO: Check that the schema matches both left and right edges schemas.
- p = Put.new(hri.getRegionName())
- p.add(HConstants::CATALOG_FAMILY,
HConstants::REGIONINFO_QUALIFIER, Writables.getBytes(hri))
- metatable.put(p)
- LOG.info("Plugged hole in .META. at: " + hri.toString())
- plugged = true
- end
- return plugged
+ hri = HRegionInfo.new(leftEdge.getTableDesc(),
leftEdge.getEndRow(), rightEdge.getStartRow())
+ p = Put.new(hri.getRegionName())
+ p.add(HConstants::CATALOG_FAMILY, HConstants::REGIONINFO_QUALIFIER,
Writables.getBytes(hri))
+ metatable.put(p)
+ LOG.info("Plugged hole in .META. at: " + hri.toString()) return true
end
fixup = isFixup()
St.Ack
Re: wrong region exception
Posted by Stack <st...@duboce.net>.
On Tue, May 31, 2011 at 3:34 PM, Robert Gonzalez
<Ro...@maxpointinteractive.com> wrote:
> The script doesn't work because it attempts to fix the hole by finding a region in the hdfs filesystem that fills the hole. But in this case there is no such file. The hole is just there.
>
OK. The fixup method has the left and right edges. Could use
previous rows regioninfo and adjust it to fill the hole (or create a
new one)? Something like the below?
diff --git a/bin/check_meta.rb b/bin/check_meta.rb
index d874922..82c6ac0 100644
--- a/bin/check_meta.rb
+++ b/bin/check_meta.rb
@@ -80,32 +80,12 @@ def getConfiguration
end
def fixup(leftEdge, rightEdge, metatable, fs, rootdir)
- plugged = nil
- # Try and fix the passed holes in meta.
- tabledir = HTableDescriptor::getTableDir(rootdir,
leftEdge.getTableDesc().getName())
- statuses = fs.listStatus(tabledir)
- for status in statuses
- next unless status.isDir()
- next if status.getPath().getName() == "compaction.dir"
- regioninfofile = Path.new(status.getPath(), ".regioninfo")
- unless fs.exists(regioninfofile)
- LOG.warn("Missing .regioninfo: " + regioninfofile.toString())
- next
- end
- is = fs.open(regioninfofile)
- hri = HRegionInfo.new()
- hri.readFields(is)
- is.close()
- next unless Bytes.equals(leftEdge.getEndKey(), hri.getStartKey())
- # TODO: Check against right edge to make sure this addition does
not overflow right edge.
- # TODO: Check that the schema matches both left and right edges schemas.
- p = Put.new(hri.getRegionName())
- p.add(HConstants::CATALOG_FAMILY,
HConstants::REGIONINFO_QUALIFIER, Writables.getBytes(hri))
- metatable.put(p)
- LOG.info("Plugged hole in .META. at: " + hri.toString())
- plugged = true
- end
- return plugged
+ hri = HRegionInfo.new(leftEdge.getTableDesc(),
leftEdge.getEndRow(), rightEdge.getStartRow())
+ p = Put.new(hri.getRegionName())
+ p.add(HConstants::CATALOG_FAMILY, HConstants::REGIONINFO_QUALIFIER,
Writables.getBytes(hri))
+ metatable.put(p)
+ LOG.info("Plugged hole in .META. at: " + hri.toString())
+ return true
end
fixup = isFixup()
St.Ack
RE: wrong region exception
Posted by Robert Gonzalez <Ro...@maxpointinteractive.com>.
The script doesn't work because it attempts to fix the hole by finding a region in the hdfs filesystem that fills the hole. But in this case there is no such file. The hole is just there.
-----Original Message-----
From: Robert Gonzalez [mailto:Robert.Gonzalez@maxpointinteractive.com]
Sent: Tuesday, May 31, 2011 5:20 PM
To: 'user@hbase.apache.org'
Subject: RE: wrong region exception
The script ran without the previous problem, but it did not fix the problem. When I ran hbck or check_meta.rb again they indicated that the problem was still there. Do I need to do something else in preparation before running check_meta?
Thanks,
Robert
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, May 31, 2011 2:57 PM
To: user@hbase.apache.org
Subject: Re: wrong region exception
Try adding this change:
Index: bin/check_meta.rb
===================================================================
--- bin/check_meta.rb (revision 1129468)
+++ bin/check_meta.rb (working copy)
@@ -127,11 +127,13 @@
scan = Scan.new()
scanner = metatable.getScanner(scan)
oldHRI = nil
-bad = nil
+bad = 0
while (result = scanner.next())
rowid = Bytes.toString(result.getRow())
rowidStr = java.lang.String.new(rowid)
bytes = result.getValue(HConstants::CATALOG_FAMILY,
HConstants::REGIONINFO_QUALIFIER)
+ next if not bytes
+ next if bytes.length == 0
hri = Writables.getHRegionInfo(bytes)
if oldHRI
if oldHRI.isOffline() && Bytes.equals(oldHRI.getStartKey(),
hri.getStartKey())
You might print out the result you have if the qualifier is null just to see what are the rows missing an HRegionInfo.
St.Ack
On Tue, May 31, 2011 at 9:02 AM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
> I'm trying my "nuclear" option: basically copy the data from the old db to a new one, skipping over bad regions. The bad news is that it is taking forever.
>
>
> I get a stack trace just trying to run check_meta.rb:
>
> maxpoint@c1-m02:/usr/lib/hbase/bin$ ./hbase org.jruby.Main
> check_meta.rb Writables.java:75:in
> `org.apache.hadoop.hbase.util.Writables.getWritable':
> java.lang.NullPointerException: null (NativeException)
> from Writables.java:119:in `org.apache.hadoop.hbase.util.Writables.getHRegionInfo'
> from NativeMethodAccessorImpl.java:-2:in `sun.reflect.NativeMethodAccessorImpl.invoke0'
> from NativeMethodAccessorImpl.java:39:in `sun.reflect.NativeMethodAccessorImpl.invoke'
> from DelegatingMethodAccessorImpl.java:25:in `sun.reflect.DelegatingMethodAccessorImpl.invoke'
> from Method.java:597:in `java.lang.reflect.Method.invoke'
> from JavaMethod.java:196:in `org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling'
> from JavaMethod.java:182:in `org.jruby.javasupport.JavaMethod.invoke_static'
> from JavaClass.java:371:in `org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute'
> ... 17 levels...
> from Main.java:183:in `org.jruby.Main.runInterpreter'
> from Main.java:120:in `org.jruby.Main.run'
> from Main.java:95:in `org.jruby.Main.main'
> Complete Java stackTrace
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75)
> at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:1
> 19)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> ava:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess
> orImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling(JavaMetho
> d.java:196)
> at
> org.jruby.javasupport.JavaMethod.invoke_static(JavaMethod.java:182)
> at
> org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute(JavaClass.
> java:371)
> at
> org.jruby.internal.runtime.methods.SimpleCallbackMethod.call(SimpleCal
> lbackMethod.java:81)
> at
> org.jruby.evaluator.EvaluationState.callNode(EvaluationState.java:571)
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 207)
> at
> org.jruby.evaluator.EvaluationState.localAsgnNode(EvaluationState.java
> :1254)
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 286)
> at
> org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533
> )
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 201)
> at
> org.jruby.evaluator.EvaluationState.whileNode(EvaluationState.java:179
> 3)
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 387)
> at
> org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533
> )
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 201)
> at
> org.jruby.evaluator.EvaluationState.rootNode(EvaluationState.java:1628
> )
> at
> org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:
> 356)
> at
> org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:164)
> at org.jruby.Ruby.eval(Ruby.java:278)
> at org.jruby.Ruby.compileOrFallbackAndRun(Ruby.java:306)
> at org.jruby.Main.runInterpreter(Main.java:238)
> at org.jruby.Main.runInterpreter(Main.java:183)
> at org.jruby.Main.run(Main.java:120)
> at org.jruby.Main.main(Main.java:95)
>
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Friday, May 27, 2011 12:43 AM
> To: user@hbase.apache.org
> Subject: Re: wrong region exception
>
> Robert:
>
> Looks like script already exists. Check bin/check_meta.rb. If you pass it --fix it should plug the hole. Read the head of the script for how to run it.
>
> Good luck,
> St.Ack
>
> On Thu, May 26, 2011 at 1:06 PM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
>> I sent the meta.txt to your saint.ack@gmail .com account due to the attachment.
>>
>> -----Original Message-----
>> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
>> Stack
>> Sent: Thursday, May 26, 2011 1:35 PM
>> To: user@hbase.apache.org
>> Subject: Re: wrong region exception
>>
>> On Thu, May 26, 2011 at 8:06 AM, Robert Gonzalez <Ro...@maxpointinteractive.com> wrote:
>>> It looks like an entire region is missing, here is the online table:
>>>
>>> urlhashv2,7FF1A5BF839C37078083B4F8267008F6,1303028235302.b0d55566fd0
>>> e
>>> 0 2ae98541618396aa7b1.
>>> c1-s03.atxd.maxpointinteractive.com:60030
>>> 7FF1A5BF839C37078083B4F8267008F6
>>> 80116D7E506D87ED39EAFFE784B5B590
>>> urlhashv2,8031483E0B3B7F587020FCBB764272D9,1305226123483.3ed065ad87f
>>> 8
>>> 9 aece6b994dd31b42b2a.
>>> c1-s33.atxd.maxpointinteractive.com:60030
>>> 8031483E0B3B7F587020FCBB764272D9
>>> 8041346D0B05617FA4B9152BFE9B18B9
>>>
>>> One ends at 80116D7E506D87ED39EAFFE784B5B590, but the next one doesn't start there.
>>>
>>
>> So, make sure you actually have a hole. Dump out your meta table:
>>
>> echo "scan '.META.'"| ./bin/hbase shell &> /tmp/meta.txt
>>
>> Then look ensure that there is a hole between the above regions (compare start and end keys... the end key of one region needs to match the start key of the next).
>>
>> If indeed a hole, you need to do a little surgery inserting a new missing region (hbck should fix this but it doesn't have the smarts just yet).
>>
>> Basically, you create a new region with start and end keys to fill the hole then you insert it into .META. and then assign it. There are some scripts in our bin directory that do various parts of this. I'm pretty sure its beyond any but a few figuring this mess out so if you do the above foot work and provide a few more details, I'll hack up something for you (and hopefully something generalized to be use by others later, and later to be integrated into hbck).
>>
>> St.Ack
>>
>