You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Lyska Anton <an...@wildec.com> on 2013/01/02 17:17:28 UTC
HConnection.locateRegions returns null
Hi all,
I want to write simple tool for rebalancing regions in cluster.
For that I need current location of all regions.
I've found method HConnection.locateRegions(tableName), but it always returns null.
Also, i've tried to call HConnection.locateRegion(regionName), and it returns null too.
sample code:
Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", host);
HConnection connection = HConnectionManager.createConnection(config);
List<HRegionLocation> locations = connection.locateRegions(tableName); //return null here
HBase Version 0.92.1-cdh4.1.2
Thanks in advance.
ERROR: disk is full, can not split region
Posted by tgh <gu...@ia.ac.cn>.
Hi
I try to use hbase and hdfs to store data, and I use 5node, and now , the disk is full, so we need to add a new node to hdfs, and now, the disk of the old nodes is full, so data node can not split for new region , and how to deal with it,
Could you help me
Thank you
-----------------------
Tian Guanhua
Re: ERROR: disk is full, can not split region
Posted by Henry JunYoung Kim <he...@gmail.com>.
Possible way to solve this problem is doing compaction manually.
It will remove datas marked deleted oermanently.
Hope to helpful.
2013. 1. 4. 오후 12:01에 "tgh" <gu...@ia.ac.cn>님이 작성:
> Hi
> I try to use hbase and hdfs to store data, and I use 5node, and
> now , the disk is full, so we need to add a new node to hdfs, and now, the
> disk of the old nodes is full, so data node can not split for new region ,
> and how to deal with it,
>
> Could you help me
>
> Thank you
> -----------------------
> Tian Guanhua
>
>
>
>
>
>
Re: HConnection.locateRegions returns null
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Nicolas,
I'm done with the update. I have implemented HRegionInfo.getStartKey
to remove the loop in locateRegion(final byte[] regionName). Tests are
running and so far, so good. I will update the JIRA when the tests
will be done.
JM
2013/1/3, Jean-Marc Spaggiari <je...@spaggiari.org>:
> I found HRegionInfo.getTableName(regionName) to get the table name
> quickly. Still searching for the startKey.
> HRegionInfo.getStartKey(regionName) doesn't exist. Maybe I will have
> to create it...
>
> 2013/1/3, Jean-Marc Spaggiari <je...@spaggiari.org>:
>> Done. You can take a look at what I pushed.
>>
>> https://issues.apache.org/jira/browse/HBASE-7488
>>
>> Regarding locateRegion(final byte[] regionName) I don't know if there
>> is a faster way to get the first row and the table name from the
>> regionName. Region name should already contain those 2 information. So
>> it should be possible to retrieve that from it instead of getting all
>> the regions and doing the comparison. Any idea if there is any util
>> class to extract the table and the key from the name?
>>
>> Thanks,
>>
>> JM
>>
>> 2013/1/3, Jean-Marc Spaggiari <je...@spaggiari.org>:
>>> I will take a look at all of that and keep you posted shortly.
>>>
>>> JM
>>>
>>> 2013/1/3, Nicolas Liochon <nk...@gmail.com>:
>>>> Yep, I'm ok with that. It will need to be put in the interface (vs. the
>>>> implementation class). Would be nice if you could implement the two
>>>> missing
>>>> methods (i.e. public HRegionLocation locateRegion(final byte []
>>>> regionName))
>>>>
>>>> On Thu, Jan 3, 2013 at 7:33 PM, Jean-Marc Spaggiari
>>>> <jean-marc@spaggiari.org
>>>>> wrote:
>>>>
>>>>> public List<HRegionLocation> locateRegions(byte[] tableName, final
>>>>
>>>
>>
>
Re: HConnection.locateRegions returns null
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
I found HRegionInfo.getTableName(regionName) to get the table name
quickly. Still searching for the startKey.
HRegionInfo.getStartKey(regionName) doesn't exist. Maybe I will have
to create it...
2013/1/3, Jean-Marc Spaggiari <je...@spaggiari.org>:
> Done. You can take a look at what I pushed.
>
> https://issues.apache.org/jira/browse/HBASE-7488
>
> Regarding locateRegion(final byte[] regionName) I don't know if there
> is a faster way to get the first row and the table name from the
> regionName. Region name should already contain those 2 information. So
> it should be possible to retrieve that from it instead of getting all
> the regions and doing the comparison. Any idea if there is any util
> class to extract the table and the key from the name?
>
> Thanks,
>
> JM
>
> 2013/1/3, Jean-Marc Spaggiari <je...@spaggiari.org>:
>> I will take a look at all of that and keep you posted shortly.
>>
>> JM
>>
>> 2013/1/3, Nicolas Liochon <nk...@gmail.com>:
>>> Yep, I'm ok with that. It will need to be put in the interface (vs. the
>>> implementation class). Would be nice if you could implement the two
>>> missing
>>> methods (i.e. public HRegionLocation locateRegion(final byte []
>>> regionName))
>>>
>>> On Thu, Jan 3, 2013 at 7:33 PM, Jean-Marc Spaggiari
>>> <jean-marc@spaggiari.org
>>>> wrote:
>>>
>>>> public List<HRegionLocation> locateRegions(byte[] tableName, final
>>>
>>
>
Re: HConnection.locateRegions returns null
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Done. You can take a look at what I pushed.
https://issues.apache.org/jira/browse/HBASE-7488
Regarding locateRegion(final byte[] regionName) I don't know if there
is a faster way to get the first row and the table name from the
regionName. Region name should already contain those 2 information. So
it should be possible to retrieve that from it instead of getting all
the regions and doing the comparison. Any idea if there is any util
class to extract the table and the key from the name?
Thanks,
JM
2013/1/3, Jean-Marc Spaggiari <je...@spaggiari.org>:
> I will take a look at all of that and keep you posted shortly.
>
> JM
>
> 2013/1/3, Nicolas Liochon <nk...@gmail.com>:
>> Yep, I'm ok with that. It will need to be put in the interface (vs. the
>> implementation class). Would be nice if you could implement the two
>> missing
>> methods (i.e. public HRegionLocation locateRegion(final byte []
>> regionName))
>>
>> On Thu, Jan 3, 2013 at 7:33 PM, Jean-Marc Spaggiari
>> <jean-marc@spaggiari.org
>>> wrote:
>>
>>> public List<HRegionLocation> locateRegions(byte[] tableName, final
>>
>
Re: HConnection.locateRegions returns null
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
I will take a look at all of that and keep you posted shortly.
JM
2013/1/3, Nicolas Liochon <nk...@gmail.com>:
> Yep, I'm ok with that. It will need to be put in the interface (vs. the
> implementation class). Would be nice if you could implement the two missing
> methods (i.e. public HRegionLocation locateRegion(final byte []
> regionName))
>
> On Thu, Jan 3, 2013 at 7:33 PM, Jean-Marc Spaggiari
> <jean-marc@spaggiari.org
>> wrote:
>
>> public List<HRegionLocation> locateRegions(byte[] tableName, final
>
Re: HConnection.locateRegions returns null
Posted by Nicolas Liochon <nk...@gmail.com>.
Yep, I'm ok with that. It will need to be put in the interface (vs. the
implementation class). Would be nice if you could implement the two missing
methods (i.e. public HRegionLocation locateRegion(final byte [] regionName))
On Thu, Jan 3, 2013 at 7:33 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:
> public List<HRegionLocation> locateRegions(byte[] tableName, final
Re: HConnection.locateRegions returns null
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
I have created HBASE-7488 to implement it.
Also, I think we should add this in the interface and do the related
implementation:
/**
* Gets the locations of all regions in the specified table, <i>tableName</i>.
* @param tableName table to get regions of
* @param offlined True if we are to include offlined regions, false and we'll
* leave out offlined regions from returned list.
* @return list of region locations for all regions of table
* @throws IOException
*/
public List<HRegionLocation> locateRegions(byte[] tableName, final
boolean offlined) throws IOException;
Because current interface doesn't allow you to specify the offlined parameter.
If you are fine with that too, I will submit the required patch.
JM
2013/1/3, Nicolas Liochon <nk...@gmail.com>:
> Hi,
>
> It will work, but there is some glue code to write as one is returning one
> region given a rowkey, while the not implemented one returns all the
> version.
> Code written by Lyska seems fine, we could put it in locateRegions (doing
> this server side is more efficient)
>
> Nicolas
>
>
> On Thu, Jan 3, 2013 at 1:34 AM, Jean-Marc Spaggiari
> <jean-marc@spaggiari.org
>> wrote:
>
>> locateRegions
>
Re: HConnection.locateRegions returns null
Posted by Nicolas Liochon <nk...@gmail.com>.
Hi,
It will work, but there is some glue code to write as one is returning one
region given a rowkey, while the not implemented one returns all the
version.
Code written by Lyska seems fine, we could put it in locateRegions (doing
this server side is more efficient)
Nicolas
On Thu, Jan 3, 2013 at 1:34 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:
> locateRegions
Re: HConnection.locateRegions returns null
Posted by Lyska Anton <an...@wildec.com>.
Hi,
thanks a lot!
as Nicolas wrote, I use
List<HRegionInfo> regions = MetaScanner.listAllRegions(config);
for (HRegionInfo info : regions) {
HRegionLocation loc = connection.locateRegion(info.getTableName(), info.getStartKey());
....
}
03.01.2013 2:34, Jean-Marc Spaggiari пишет:
> Here is the code for locateRegion:
>
> @Override
> public List<HRegionLocation> locateRegions(final byte [] tableName)
> throws IOException {
> // TODO implement. use old stuff or new stuff?
> return null;
> }
>
>
> As you can see, it's like what Nicolas is saying. Always return null.
>
> However, relocateRegion is calling a private locateRegions which is
> not returning null.... You might want to try that?
>
> Also, I played few weeks ago with the region balancing. You can take
> a look here to see what I did.
> http://www.spaggiari.org/index.php/hbase/changing-the-hbase-default-loadbalancer
> You can do yours based on that.
>
> Nicolas, should we not have locateRegion "simply" calling the private
> locateRegion method?
>
> JM
>
> 2013/1/2, Nicolas Liochon <nk...@gmail.com>:
>> Hi,
>>
>> It's actually not implemented (I will update the javadoc for the latest
>> version)
>> You can use locateRegion(final byte [] tableName, final byte [] row), and
>> iterates on all region start keys to get all the regions.
>>
>> Cheers,
>>
>> Nicolas
>>
>>
>>
>>
>> On Wed, Jan 2, 2013 at 5:17 PM, Lyska Anton <an...@wildec.com> wrote:
>>
>>> Hi all,
>>>
>>> I want to write simple tool for rebalancing regions in cluster.
>>> For that I need current location of all regions.
>>> I've found method HConnection.locateRegions(**tableName), but it always
>>> returns null.
>>> Also, i've tried to call HConnection.locateRegion(**regionName), and it
>>> returns null too.
>>>
>>> sample code:
>>>
>>> Configuration config = HBaseConfiguration.create();
>>> config.set("hbase.zookeeper.**quorum", host);
>>>
>>> HConnection connection = HConnectionManager.**
>>> createConnection(config);
>>> List<HRegionLocation> locations =
>>> connection.locateRegions(**tableName);
>>> //return null here
>>>
>>> HBase Version 0.92.1-cdh4.1.2
>>>
>>> Thanks in advance.
>>>
>>>
Re: HConnection.locateRegions returns null
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Here is the code for locateRegion:
@Override
public List<HRegionLocation> locateRegions(final byte [] tableName)
throws IOException {
// TODO implement. use old stuff or new stuff?
return null;
}
As you can see, it's like what Nicolas is saying. Always return null.
However, relocateRegion is calling a private locateRegions which is
not returning null.... You might want to try that?
Also, I played few weeks ago with the region balancing. You can take
a look here to see what I did.
http://www.spaggiari.org/index.php/hbase/changing-the-hbase-default-loadbalancer
You can do yours based on that.
Nicolas, should we not have locateRegion "simply" calling the private
locateRegion method?
JM
2013/1/2, Nicolas Liochon <nk...@gmail.com>:
> Hi,
>
> It's actually not implemented (I will update the javadoc for the latest
> version)
> You can use locateRegion(final byte [] tableName, final byte [] row), and
> iterates on all region start keys to get all the regions.
>
> Cheers,
>
> Nicolas
>
>
>
>
> On Wed, Jan 2, 2013 at 5:17 PM, Lyska Anton <an...@wildec.com> wrote:
>
>> Hi all,
>>
>> I want to write simple tool for rebalancing regions in cluster.
>> For that I need current location of all regions.
>> I've found method HConnection.locateRegions(**tableName), but it always
>> returns null.
>> Also, i've tried to call HConnection.locateRegion(**regionName), and it
>> returns null too.
>>
>> sample code:
>>
>> Configuration config = HBaseConfiguration.create();
>> config.set("hbase.zookeeper.**quorum", host);
>>
>> HConnection connection = HConnectionManager.**
>> createConnection(config);
>> List<HRegionLocation> locations =
>> connection.locateRegions(**tableName);
>> //return null here
>>
>> HBase Version 0.92.1-cdh4.1.2
>>
>> Thanks in advance.
>>
>>
>
Re: HConnection.locateRegions returns null
Posted by Nicolas Liochon <nk...@gmail.com>.
Hi,
It's actually not implemented (I will update the javadoc for the latest
version)
You can use locateRegion(final byte [] tableName, final byte [] row), and
iterates on all region start keys to get all the regions.
Cheers,
Nicolas
On Wed, Jan 2, 2013 at 5:17 PM, Lyska Anton <an...@wildec.com> wrote:
> Hi all,
>
> I want to write simple tool for rebalancing regions in cluster.
> For that I need current location of all regions.
> I've found method HConnection.locateRegions(**tableName), but it always
> returns null.
> Also, i've tried to call HConnection.locateRegion(**regionName), and it
> returns null too.
>
> sample code:
>
> Configuration config = HBaseConfiguration.create();
> config.set("hbase.zookeeper.**quorum", host);
>
> HConnection connection = HConnectionManager.**
> createConnection(config);
> List<HRegionLocation> locations = connection.locateRegions(**tableName);
> //return null here
>
> HBase Version 0.92.1-cdh4.1.2
>
> Thanks in advance.
>
>