You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by vamshi krishna <va...@gmail.com> on 2011/08/18 05:05:10 UTC
Where are .META. and ROOT tables data
Hi all, i want to know the actual location of .META. and ROOT data after we
install HBase in our machine. i mean, can i directly see that data in a
particular file or something like that unlike querying and getting some
specific data from them??
And the most important doubt is, which version of hadoop i should use to run
HBase at current time.. means asper latest versions which versions are
compatible?
*
Thank you..
Vamshi Krishna
*
Re: Where are .META. and ROOT tables data
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Don't worry Lars you got it right :)
I might also add that we don't use a DHT for the same reason the user
tables don't use a DHT.
And finally ROOT and META regions tend not to move too much, so once
their location is cached you never need to ask their parent where to
find them. Even if they do move, their actual content is also often
cached client-side so they don't even git hit that much (unless you
are using short-lived client).
J-D
On Mon, Aug 29, 2011 at 9:00 AM, lars hofhansl <lh...@yahoo.com> wrote:
> There are folks who know this better than me, but here's my understanding:
>
> .META. is consulted to find a region of a table for a specific row key.
> So only after consulting .META. do you know which regionserver handles your key.
>
> Now, how do you find regions in .META. itself (So you can consult it to find the correct region of your table)?
> In order to find the right regionserver you... first have to know the right regionserver... It's a chicken and egg problem.
>
> That's where -ROOT- comes into play. -ROOT- always has exactly one region and its location is known.
>
> .META. is (currently) always one region (so in that respect -ROOT- is not *currently* needed, but that will probably change).
>
> Neither -ROOT- not .META. are actually single points of failure. If their regionserver dies they'll be re-assigned. It might just take a little bit.
>
> Stack, please chime in, if I didn't get this right :)
>
> -- Lars
>
>
> ----- Original Message -----
> From: vamshi krishna <va...@gmail.com>
> To: user@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
> Cc:
> Sent: Sunday, August 28, 2011 11:52 PM
> Subject: Re: Where are .META. and ROOT tables data
>
> Hi lars and folks,
>
> Can anyone please explain the exact purpose of having -ROOT- except storing
> the locations of the regions in .META. (meta data table) ? I could not find
> the special use and importance of -ROOT- in HBase anywhere.
>
> Even without ROOT being stored in a particular region of HBase , we can
> have the .META. itself distributed across RSs and client requests be
> redirected to any Region Server , which is chosen randomly. Then using the
> Distributed Hash Table we can resolve the client's requesting location i.e
> which RS will hold the region responsible for location. Now the clients
> request is actually handled by the resultant RS.
> The *main idea* being is , we can get rid off the single point failure
> with respect to -ROOT- stored in a single region.
>
> On Thu, Aug 18, 2011 at 12:13 PM, vamshi krishna <va...@gmail.com>wrote:
>
>> Thank you very much Lars..!
>>
>>
>> On Thu, Aug 18, 2011 at 10:30 AM, lars hofhansl <lh...@yahoo.com>wrote:
>>
>>> Hi Vamshi,
>>>
>>> at this point HBase needs a version of Hadoop that did not have a stable
>>> release, yet.
>>>
>>>
>>> Check out:http://hbase.apache.org/book/notsoquick.html For more details.
>>> We are using the CDH3 distribution and it works very well so far.
>>> We also have successfully used custom builds of HBase trunk against the
>>> standard CDH3 Hadoop version, by just replacing the hadoop-xxx.jar in
>>> hbase-xxx/lib.
>>>
>>>
>>> When you configured HBase you defined a root directory in hbase-site.xml
>>> (if not, see above link), that's where you will find META and ROOT tables.
>>>
>>> (The root directory would be a local directory in local mode, or a
>>> directory in HDFS).
>>>
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ________________________________
>>> From: vamshi krishna <va...@gmail.com>
>>> To: user@hbase.apache.org
>>> Sent: Wednesday, August 17, 2011 8:05 PM
>>> Subject: Where are .META. and ROOT tables data
>>>
>>> Hi all, i want to know the actual location of .META. and ROOT data after
>>> we
>>> install HBase in our machine. i mean, can i directly see that data in a
>>> particular file or something like that unlike querying and getting some
>>> specific data from them??
>>> And the most important doubt is, which version of hadoop i should use to
>>> run
>>> HBase at current time.. means asper latest versions which versions are
>>> compatible?
>>> *
>>> Thank you..
>>>
>>>
>>> Vamshi Krishna
>>> *
>>
>>
>>
>>
>> --
>> *Regards*
>> *
>> Vamshi Krishna
>> *
>>
>
>
>
> --
> *Regards*
> *
> Vamshi Krishna
> *
>
>
Re: Where are .META. and ROOT tables data
Posted by lars hofhansl <lh...@yahoo.com>.
There are folks who know this better than me, but here's my understanding:
.META. is consulted to find a region of a table for a specific row key.
So only after consulting .META. do you know which regionserver handles your key.
Now, how do you find regions in .META. itself (So you can consult it to find the correct region of your table)?
In order to find the right regionserver you... first have to know the right regionserver... It's a chicken and egg problem.
That's where -ROOT- comes into play. -ROOT- always has exactly one region and its location is known.
.META. is (currently) always one region (so in that respect -ROOT- is not *currently* needed, but that will probably change).
Neither -ROOT- not .META. are actually single points of failure. If their regionserver dies they'll be re-assigned. It might just take a little bit.
Stack, please chime in, if I didn't get this right :)
-- Lars
----- Original Message -----
From: vamshi krishna <va...@gmail.com>
To: user@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
Cc:
Sent: Sunday, August 28, 2011 11:52 PM
Subject: Re: Where are .META. and ROOT tables data
Hi lars and folks,
Can anyone please explain the exact purpose of having -ROOT- except storing
the locations of the regions in .META. (meta data table) ? I could not find
the special use and importance of -ROOT- in HBase anywhere.
Even without ROOT being stored in a particular region of HBase , we can
have the .META. itself distributed across RSs and client requests be
redirected to any Region Server , which is chosen randomly. Then using the
Distributed Hash Table we can resolve the client's requesting location i.e
which RS will hold the region responsible for location. Now the clients
request is actually handled by the resultant RS.
The *main idea* being is , we can get rid off the single point failure
with respect to -ROOT- stored in a single region.
On Thu, Aug 18, 2011 at 12:13 PM, vamshi krishna <va...@gmail.com>wrote:
> Thank you very much Lars..!
>
>
> On Thu, Aug 18, 2011 at 10:30 AM, lars hofhansl <lh...@yahoo.com>wrote:
>
>> Hi Vamshi,
>>
>> at this point HBase needs a version of Hadoop that did not have a stable
>> release, yet.
>>
>>
>> Check out:http://hbase.apache.org/book/notsoquick.html For more details.
>> We are using the CDH3 distribution and it works very well so far.
>> We also have successfully used custom builds of HBase trunk against the
>> standard CDH3 Hadoop version, by just replacing the hadoop-xxx.jar in
>> hbase-xxx/lib.
>>
>>
>> When you configured HBase you defined a root directory in hbase-site.xml
>> (if not, see above link), that's where you will find META and ROOT tables.
>>
>> (The root directory would be a local directory in local mode, or a
>> directory in HDFS).
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>> From: vamshi krishna <va...@gmail.com>
>> To: user@hbase.apache.org
>> Sent: Wednesday, August 17, 2011 8:05 PM
>> Subject: Where are .META. and ROOT tables data
>>
>> Hi all, i want to know the actual location of .META. and ROOT data after
>> we
>> install HBase in our machine. i mean, can i directly see that data in a
>> particular file or something like that unlike querying and getting some
>> specific data from them??
>> And the most important doubt is, which version of hadoop i should use to
>> run
>> HBase at current time.. means asper latest versions which versions are
>> compatible?
>> *
>> Thank you..
>>
>>
>> Vamshi Krishna
>> *
>
>
>
>
> --
> *Regards*
> *
> Vamshi Krishna
> *
>
--
*Regards*
*
Vamshi Krishna
*
Re: Where are .META. and ROOT tables data
Posted by vamshi krishna <va...@gmail.com>.
Hi lars and folks,
Can anyone please explain the exact purpose of having -ROOT- except storing
the locations of the regions in .META. (meta data table) ? I could not find
the special use and importance of -ROOT- in HBase anywhere.
Even without ROOT being stored in a particular region of HBase , we can
have the .META. itself distributed across RSs and client requests be
redirected to any Region Server , which is chosen randomly. Then using the
Distributed Hash Table we can resolve the client's requesting location i.e
which RS will hold the region responsible for location. Now the clients
request is actually handled by the resultant RS.
The *main idea* being is , we can get rid off the single point failure
with respect to -ROOT- stored in a single region.
On Thu, Aug 18, 2011 at 12:13 PM, vamshi krishna <va...@gmail.com>wrote:
> Thank you very much Lars..!
>
>
> On Thu, Aug 18, 2011 at 10:30 AM, lars hofhansl <lh...@yahoo.com>wrote:
>
>> Hi Vamshi,
>>
>> at this point HBase needs a version of Hadoop that did not have a stable
>> release, yet.
>>
>>
>> Check out:http://hbase.apache.org/book/notsoquick.html For more details.
>> We are using the CDH3 distribution and it works very well so far.
>> We also have successfully used custom builds of HBase trunk against the
>> standard CDH3 Hadoop version, by just replacing the hadoop-xxx.jar in
>> hbase-xxx/lib.
>>
>>
>> When you configured HBase you defined a root directory in hbase-site.xml
>> (if not, see above link), that's where you will find META and ROOT tables.
>>
>> (The root directory would be a local directory in local mode, or a
>> directory in HDFS).
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>> From: vamshi krishna <va...@gmail.com>
>> To: user@hbase.apache.org
>> Sent: Wednesday, August 17, 2011 8:05 PM
>> Subject: Where are .META. and ROOT tables data
>>
>> Hi all, i want to know the actual location of .META. and ROOT data after
>> we
>> install HBase in our machine. i mean, can i directly see that data in a
>> particular file or something like that unlike querying and getting some
>> specific data from them??
>> And the most important doubt is, which version of hadoop i should use to
>> run
>> HBase at current time.. means asper latest versions which versions are
>> compatible?
>> *
>> Thank you..
>>
>>
>> Vamshi Krishna
>> *
>
>
>
>
> --
> *Regards*
> *
> Vamshi Krishna
> *
>
--
*Regards*
*
Vamshi Krishna
*
Re: Where are .META. and ROOT tables data
Posted by vamshi krishna <va...@gmail.com>.
Thank you very much Lars..!
On Thu, Aug 18, 2011 at 10:30 AM, lars hofhansl <lh...@yahoo.com> wrote:
> Hi Vamshi,
>
> at this point HBase needs a version of Hadoop that did not have a stable
> release, yet.
>
>
> Check out:http://hbase.apache.org/book/notsoquick.html For more details.
> We are using the CDH3 distribution and it works very well so far.
> We also have successfully used custom builds of HBase trunk against the
> standard CDH3 Hadoop version, by just replacing the hadoop-xxx.jar in
> hbase-xxx/lib.
>
>
> When you configured HBase you defined a root directory in hbase-site.xml
> (if not, see above link), that's where you will find META and ROOT tables.
>
> (The root directory would be a local directory in local mode, or a
> directory in HDFS).
>
>
> -- Lars
>
>
>
> ________________________________
> From: vamshi krishna <va...@gmail.com>
> To: user@hbase.apache.org
> Sent: Wednesday, August 17, 2011 8:05 PM
> Subject: Where are .META. and ROOT tables data
>
> Hi all, i want to know the actual location of .META. and ROOT data after we
> install HBase in our machine. i mean, can i directly see that data in a
> particular file or something like that unlike querying and getting some
> specific data from them??
> And the most important doubt is, which version of hadoop i should use to
> run
> HBase at current time.. means asper latest versions which versions are
> compatible?
> *
> Thank you..
>
>
> Vamshi Krishna
> *
--
*Regards*
*
Vamshi Krishna
*
Re: Where are .META. and ROOT tables data
Posted by lars hofhansl <lh...@yahoo.com>.
Hi Vamshi,
at this point HBase needs a version of Hadoop that did not have a stable release, yet.
Check out:http://hbase.apache.org/book/notsoquick.html For more details.
We are using the CDH3 distribution and it works very well so far.
We also have successfully used custom builds of HBase trunk against the standard CDH3 Hadoop version, by just replacing the hadoop-xxx.jar in hbase-xxx/lib.
When you configured HBase you defined a root directory in hbase-site.xml (if not, see above link), that's where you will find META and ROOT tables.
(The root directory would be a local directory in local mode, or a directory in HDFS).
-- Lars
________________________________
From: vamshi krishna <va...@gmail.com>
To: user@hbase.apache.org
Sent: Wednesday, August 17, 2011 8:05 PM
Subject: Where are .META. and ROOT tables data
Hi all, i want to know the actual location of .META. and ROOT data after we
install HBase in our machine. i mean, can i directly see that data in a
particular file or something like that unlike querying and getting some
specific data from them??
And the most important doubt is, which version of hadoop i should use to run
HBase at current time.. means asper latest versions which versions are
compatible?
*
Thank you..
Vamshi Krishna
*