You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by yuliya Feldman <yu...@yahoo.com> on 2010/10/08 03:41:31 UTC
Question regarding regions move/assignement
Hello here,
I have a question regarding HBase data locality in respect to HDFS data nodes.
As far as I understand regions created by region servers will happen to be
physically created on local Data Node if RegionServer happen to run on one.
I had an issue with writing to Meta table and my table "dissapeared" from list
of tables. After I restarted hbase everything went back to normal, but regions
got reassigned to different region servers they were originally created.
How this "(re)assignment" mechanism actually works?
I am using hbase 0.20.4 at the moment.
Thanks
Re: Question regarding regions move/assignement
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Yes, it does, until compactions happen.
J-D
On Fri, Oct 8, 2010 at 3:39 PM, yuliya Feldman <yu...@yahoo.com> wrote:
> I understand that data will be fetched via network, my concern is that instead
> of assigning local regions to regionserver it assigns remote ones which means
> that all data access will be remote
>
>
>
> ----- Original Message ----
> From: Jean-Daniel Cryans <jd...@apache.org>
> To: user@hbase.apache.org
> Sent: Fri, October 8, 2010 3:28:15 PM
> Subject: Re: Question regarding regions move/assignement
>
> Even with rep=1, HDFS appears like one single filesystem to HBase so
> the blocks that are remote to the region server will be fetched via
> network.
>
> J-D
>
> On Fri, Oct 8, 2010 at 3:25 PM, yuliya Feldman <yu...@yahoo.com> wrote:
>> Thanks for the link. It is very helpful.
>>
>> My concerns was that after HBase restart (table "dissapears" means - looks
> like
>> some data got corrupted and HBase stopped showing my table in the list of
>> Tables) some regions that were originally created on one region server got
>> reassigned to different regionserver after restart, although physically data
>> still on data node of first region server.
>>
>> My replication was 1, so data belongs to just a single data node.
>>
>> Thanks
>>
>>
>>
>> ----- Original Message ----
>> From: Jean-Daniel Cryans <jd...@apache.org>
>> To: user@hbase.apache.org
>> Sent: Fri, October 8, 2010 2:49:18 PM
>> Subject: Re: Question regarding regions move/assignement
>>
>> Not sure about the part where your table "disappears", but your
>> question is mostly answered in this blog post by our good friend Lars
>> G. http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html
>>
>> J-D
>>
>> On Thu, Oct 7, 2010 at 6:41 PM, yuliya Feldman <yu...@yahoo.com> wrote:
>>> Hello here,
>>>
>>> I have a question regarding HBase data locality in respect to HDFS data
> nodes.
>>> As far as I understand regions created by region servers will happen to be
>>> physically created on local Data Node if RegionServer happen to run on one.
>>>
>>> I had an issue with writing to Meta table and my table "dissapeared" from
> list
>>> of tables. After I restarted hbase everything went back to normal, but
> regions
>>> got reassigned to different region servers they were originally created.
>>>
>>> How this "(re)assignment" mechanism actually works?
>>>
>>> I am using hbase 0.20.4 at the moment.
>>>
>>> Thanks
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
Re: Question regarding regions move/assignement
Posted by yuliya Feldman <yu...@yahoo.com>.
I understand that data will be fetched via network, my concern is that instead
of assigning local regions to regionserver it assigns remote ones which means
that all data access will be remote
----- Original Message ----
From: Jean-Daniel Cryans <jd...@apache.org>
To: user@hbase.apache.org
Sent: Fri, October 8, 2010 3:28:15 PM
Subject: Re: Question regarding regions move/assignement
Even with rep=1, HDFS appears like one single filesystem to HBase so
the blocks that are remote to the region server will be fetched via
network.
J-D
On Fri, Oct 8, 2010 at 3:25 PM, yuliya Feldman <yu...@yahoo.com> wrote:
> Thanks for the link. It is very helpful.
>
> My concerns was that after HBase restart (table "dissapears" means - looks
like
> some data got corrupted and HBase stopped showing my table in the list of
> Tables) some regions that were originally created on one region server got
> reassigned to different regionserver after restart, although physically data
> still on data node of first region server.
>
> My replication was 1, so data belongs to just a single data node.
>
> Thanks
>
>
>
> ----- Original Message ----
> From: Jean-Daniel Cryans <jd...@apache.org>
> To: user@hbase.apache.org
> Sent: Fri, October 8, 2010 2:49:18 PM
> Subject: Re: Question regarding regions move/assignement
>
> Not sure about the part where your table "disappears", but your
> question is mostly answered in this blog post by our good friend Lars
> G. http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html
>
> J-D
>
> On Thu, Oct 7, 2010 at 6:41 PM, yuliya Feldman <yu...@yahoo.com> wrote:
>> Hello here,
>>
>> I have a question regarding HBase data locality in respect to HDFS data
nodes.
>> As far as I understand regions created by region servers will happen to be
>> physically created on local Data Node if RegionServer happen to run on one.
>>
>> I had an issue with writing to Meta table and my table "dissapeared" from
list
>> of tables. After I restarted hbase everything went back to normal, but
regions
>> got reassigned to different region servers they were originally created.
>>
>> How this "(re)assignment" mechanism actually works?
>>
>> I am using hbase 0.20.4 at the moment.
>>
>> Thanks
>>
>>
>>
>>
>>
>
>
>
>
>
Re: Question regarding regions move/assignement
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Even with rep=1, HDFS appears like one single filesystem to HBase so
the blocks that are remote to the region server will be fetched via
network.
J-D
On Fri, Oct 8, 2010 at 3:25 PM, yuliya Feldman <yu...@yahoo.com> wrote:
> Thanks for the link. It is very helpful.
>
> My concerns was that after HBase restart (table "dissapears" means - looks like
> some data got corrupted and HBase stopped showing my table in the list of
> Tables) some regions that were originally created on one region server got
> reassigned to different regionserver after restart, although physically data
> still on data node of first region server.
>
> My replication was 1, so data belongs to just a single data node.
>
> Thanks
>
>
>
> ----- Original Message ----
> From: Jean-Daniel Cryans <jd...@apache.org>
> To: user@hbase.apache.org
> Sent: Fri, October 8, 2010 2:49:18 PM
> Subject: Re: Question regarding regions move/assignement
>
> Not sure about the part where your table "disappears", but your
> question is mostly answered in this blog post by our good friend Lars
> G. http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html
>
> J-D
>
> On Thu, Oct 7, 2010 at 6:41 PM, yuliya Feldman <yu...@yahoo.com> wrote:
>> Hello here,
>>
>> I have a question regarding HBase data locality in respect to HDFS data nodes.
>> As far as I understand regions created by region servers will happen to be
>> physically created on local Data Node if RegionServer happen to run on one.
>>
>> I had an issue with writing to Meta table and my table "dissapeared" from list
>> of tables. After I restarted hbase everything went back to normal, but regions
>> got reassigned to different region servers they were originally created.
>>
>> How this "(re)assignment" mechanism actually works?
>>
>> I am using hbase 0.20.4 at the moment.
>>
>> Thanks
>>
>>
>>
>>
>>
>
>
>
>
>
Re: Question regarding regions move/assignement
Posted by yuliya Feldman <yu...@yahoo.com>.
Thanks for the link. It is very helpful.
My concerns was that after HBase restart (table "dissapears" means - looks like
some data got corrupted and HBase stopped showing my table in the list of
Tables) some regions that were originally created on one region server got
reassigned to different regionserver after restart, although physically data
still on data node of first region server.
My replication was 1, so data belongs to just a single data node.
Thanks
----- Original Message ----
From: Jean-Daniel Cryans <jd...@apache.org>
To: user@hbase.apache.org
Sent: Fri, October 8, 2010 2:49:18 PM
Subject: Re: Question regarding regions move/assignement
Not sure about the part where your table "disappears", but your
question is mostly answered in this blog post by our good friend Lars
G. http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html
J-D
On Thu, Oct 7, 2010 at 6:41 PM, yuliya Feldman <yu...@yahoo.com> wrote:
> Hello here,
>
> I have a question regarding HBase data locality in respect to HDFS data nodes.
> As far as I understand regions created by region servers will happen to be
> physically created on local Data Node if RegionServer happen to run on one.
>
> I had an issue with writing to Meta table and my table "dissapeared" from list
> of tables. After I restarted hbase everything went back to normal, but regions
> got reassigned to different region servers they were originally created.
>
> How this "(re)assignment" mechanism actually works?
>
> I am using hbase 0.20.4 at the moment.
>
> Thanks
>
>
>
>
>
Re: Question regarding regions move/assignement
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Not sure about the part where your table "disappears", but your
question is mostly answered in this blog post by our good friend Lars
G. http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html
J-D
On Thu, Oct 7, 2010 at 6:41 PM, yuliya Feldman <yu...@yahoo.com> wrote:
> Hello here,
>
> I have a question regarding HBase data locality in respect to HDFS data nodes.
> As far as I understand regions created by region servers will happen to be
> physically created on local Data Node if RegionServer happen to run on one.
>
> I had an issue with writing to Meta table and my table "dissapeared" from list
> of tables. After I restarted hbase everything went back to normal, but regions
> got reassigned to different region servers they were originally created.
>
> How this "(re)assignment" mechanism actually works?
>
> I am using hbase 0.20.4 at the moment.
>
> Thanks
>
>
>
>
>