You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by John Edstrom <je...@nearinfinity.com> on 2012/09/20 19:33:11 UTC
IllegalArgumentException when trying to split an empty region
Hello,
We're having a problem when calling HBaseAdmin.split() on an empty region.
The new regions are never getting created and the RegionServer appears to
be throwing an IllegalArgumentException with the following stack trace:
java.lang.IllegalArgumentException
at
java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:589)
at
java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:514)
2631 at
java.util.concurrent.Executors.newFixedThreadPool(Executors.java:84)
2632 at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:592)
2633 at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:290)
2634 at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:452)
2635 at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
2636 at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
2637 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
2638 at java.lang.Thread.run(Thread.java:680)
RE: IllegalArgumentException when trying to split an empty region
Posted by "Ramkrishna.S.Vasudevan" <ra...@huawei.com>.
Hi John
We have encountered this problem while developing one of the features.
I suggest we can raise an issue in JIRA.
The problem here is there is a thread pool executor that tries to split the
store files and it takes the no of store files as the no of threads needed
for the executor.
So for an empty region the no of store files is going to be 0 anyway.
Regards
Ram
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Friday, September 21, 2012 10:07 AM
> To: user@hbase.apache.org
> Subject: Re: IllegalArgumentException when trying to split an empty
> region
>
> On Thu, Sep 20, 2012 at 10:36 AM, John Edstrom
> <je...@nearinfinity.com> wrote:
> > Apologies, this sent before I had finished writing it :X
> >
> > The stack trace is below, but what we are attempting to do is load
> data
> > into HBase via MapReduce. When we're doing the load, we write the
> HFiles
> > using HFileOutputFormat and then bulk load them into HBase. At the
> start of
> > the MapReduce, we don't have the region splits available to us, so
> all our
> > data is getting written to a handful of regions. Therefore, we are
> > attempting to manually split the regions before the MapReduce job so
> that
> > the HFiles will be evenly written across many regions.
> >
> > Any help or guidance would be appreciated.
> >
>
> What version John?
>
> Could you use this method when you create the table:
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdm
> in.html#createTable(org.apache.hadoop.hbase.HTableDescriptor,
> byte[][])
>
> St.Ack
RE: IllegalArgumentException when trying to split an empty region
Posted by "Ramkrishna.S.Vasudevan" <ra...@huawei.com>.
Raised HBASE-6853 to address this issue.
Regards
Ram
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Friday, September 21, 2012 10:07 AM
> To: user@hbase.apache.org
> Subject: Re: IllegalArgumentException when trying to split an empty
> region
>
> On Thu, Sep 20, 2012 at 10:36 AM, John Edstrom
> <je...@nearinfinity.com> wrote:
> > Apologies, this sent before I had finished writing it :X
> >
> > The stack trace is below, but what we are attempting to do is load
> data
> > into HBase via MapReduce. When we're doing the load, we write the
> HFiles
> > using HFileOutputFormat and then bulk load them into HBase. At the
> start of
> > the MapReduce, we don't have the region splits available to us, so
> all our
> > data is getting written to a handful of regions. Therefore, we are
> > attempting to manually split the regions before the MapReduce job so
> that
> > the HFiles will be evenly written across many regions.
> >
> > Any help or guidance would be appreciated.
> >
>
> What version John?
>
> Could you use this method when you create the table:
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdm
> in.html#createTable(org.apache.hadoop.hbase.HTableDescriptor,
> byte[][])
>
> St.Ack
Re: IllegalArgumentException when trying to split an empty region
Posted by Stack <st...@duboce.net>.
On Thu, Sep 20, 2012 at 10:36 AM, John Edstrom
<je...@nearinfinity.com> wrote:
> Apologies, this sent before I had finished writing it :X
>
> The stack trace is below, but what we are attempting to do is load data
> into HBase via MapReduce. When we're doing the load, we write the HFiles
> using HFileOutputFormat and then bulk load them into HBase. At the start of
> the MapReduce, we don't have the region splits available to us, so all our
> data is getting written to a handful of regions. Therefore, we are
> attempting to manually split the regions before the MapReduce job so that
> the HFiles will be evenly written across many regions.
>
> Any help or guidance would be appreciated.
>
What version John?
Could you use this method when you create the table:
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#createTable(org.apache.hadoop.hbase.HTableDescriptor,
byte[][])
St.Ack
Re: IllegalArgumentException when trying to split an empty region
Posted by John Edstrom <je...@nearinfinity.com>.
Apologies, this sent before I had finished writing it :X
The stack trace is below, but what we are attempting to do is load data
into HBase via MapReduce. When we're doing the load, we write the HFiles
using HFileOutputFormat and then bulk load them into HBase. At the start of
the MapReduce, we don't have the region splits available to us, so all our
data is getting written to a handful of regions. Therefore, we are
attempting to manually split the regions before the MapReduce job so that
the HFiles will be evenly written across many regions.
Any help or guidance would be appreciated.
Thanks,
John
On Thu, Sep 20, 2012 at 1:33 PM, John Edstrom <je...@nearinfinity.com>wrote:
> Hello,
>
> We're having a problem when calling HBaseAdmin.split() on an empty region.
> The new regions are never getting created and the RegionServer appears to
> be throwing an IllegalArgumentException with the following stack trace:
> java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:589)
> at
> java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:514)
> 2631 at
> java.util.concurrent.Executors.newFixedThreadPool(Executors.java:84)
> 2632 at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:592)
> 2633 at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:290)
> 2634 at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:452)
> 2635 at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
> 2636 at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 2637 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 2638 at java.lang.Thread.run(Thread.java:680)
>
RE: IllegalArgumentException when trying to split an empty region
Posted by Anoop Sam John <an...@huawei.com>.
>We're having a problem when calling HBaseAdmin.split() on an empty region.
You are calling split on a region with out passing any split point right?
Yes in that case if there are no data in the region it wont be able to do the split. In such a case the split point is calculated based on the max, min rowkeys in that region..
Can u pass the split key in your case?
-Anoop-
________________________________________
From: John Edstrom [jedstrom@nearinfinity.com]
Sent: Thursday, September 20, 2012 11:03 PM
To: user@hbase.apache.org
Subject: IllegalArgumentException when trying to split an empty region
Hello,
We're having a problem when calling HBaseAdmin.split() on an empty region.
The new regions are never getting created and the RegionServer appears to
be throwing an IllegalArgumentException with the following stack trace:
java.lang.IllegalArgumentException
at
java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:589)
at
java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:514)
2631 at
java.util.concurrent.Executors.newFixedThreadPool(Executors.java:84)
2632 at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:592)
2633 at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:290)
2634 at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:452)
2635 at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
2636 at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
2637 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
2638 at java.lang.Thread.run(Thread.java:680)