You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by William Kang <we...@gmail.com> on 2010/03/01 21:36:14 UTC

Questions about HBase

Hi guys,
I am new to HBase and have several questions. Would anybody kindly answer
some of them?

1. Why HBase could provide a low-latency random access to files compared to
HDFS?

2. By default, Only a single row at a time may be locked. Is it a single
client who can only lock one or is it globally can only lock one?  If this
is the case, by default, will the performance be really bad?

Many thanks!


William

Re: Questions about HBase

Posted by Erik Holstad <er...@gmail.com>.
Hey William!

On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>wrote:

> Hi guys,
> I am new to HBase and have several questions. Would anybody kindly answer
> some of them?
>
> 1. Why HBase could provide a low-latency random access to files compared to
> HDFS?
>
Have a look at http://wiki.apache.org/hadoop/Hbase and the bigtable paper
for reference on how
to add random access on top of a dfs.


>
> 2. By default, Only a single row at a time may be locked. Is it a single
> client who can only lock one or is it globally can only lock one?  If this
> is the case, by default, will the performance be really bad?
>
The lock is global to insure that a for example read/update/put action can
take
place. Performance will suffer if you have a lot of clients doing this to
the same
row a the same time. In my experience this is not really that common, but if
this
is something you need have a look at the  ITHBase.


> Many thanks!
>
>
> William
>



-- 
Regards Erik

Re: Questions about HBase

Posted by Stack <st...@duboce.net>.
On Mon, Mar 1, 2010 at 6:13 PM, William Kang <we...@gmail.com> wrote:
> Hi Erik and Ryan,
> Thanks a lot for your replies.
> I think I probably misunderstood something since I saw in HBase
> documentation it says "MapFiles cannot currently be mapped into memory". But
> the index of it can still be mapped into memory, right? Is it just like
> bigtable that the index is at the end of SSTable or else?
>

Yes.

> And about the row lock question, if I have many users need pull out the same
> row (read), what gonna happen?
>

They'll pile up on the lock (But as Erik says, this should be rare,
and if row is hot, it'll be in cache, etc.)


> And another question, why the ROOT table and META tables are not in the
> HBaseMaster's memory? Would this greatly degrade the performance?
> Thanks.
>

The META table at least can get big if you have a big cluster with
loads of data loaded.

The META table will likely be mostly served from cache on hosting
servers since its usually frequently accesed.

ST.Ack





> Thanks.
>
>
> Cao
>
> On Mon, Mar 1, 2010 at 5:33 PM, Erik Holstad <er...@gmail.com> wrote:
>
>> On Mon, Mar 1, 2010 at 2:16 PM, Ryan Rawson <ry...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > 1.  We use in-memory indexes to get fast random reads.  Our index
>> > tells us to read block X of a file only retrieving a small amount of
>> > the file to satisfy the user's read.
>> >
>> > 2.  The row locking is not global - for each row there can only be 1
>> > thread doing a put at a time.  This serializes all puts to a single
>> > row. It is NOT global.
>> >
>>
>> So, this is me not understanding the difference between global locking and
>> waiting in a line until the tasks ahead of you are done :)
>>
>> >
>> > On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>
>> > wrote:
>> > > Hi guys,
>> > > I am new to HBase and have several questions. Would anybody kindly
>> answer
>> > > some of them?
>> > >
>> > > 1. Why HBase could provide a low-latency random access to files
>> compared
>> > to
>> > > HDFS?
>> > >
>> > > 2. By default, Only a single row at a time may be locked. Is it a
>> single
>> > > client who can only lock one or is it globally can only lock one?  If
>> > this
>> > > is the case, by default, will the performance be really bad?
>> > >
>> > > Many thanks!
>> > >
>> > >
>> > > William
>> > >
>> >
>>
>>
>>
>> --
>> Regards Erik
>>
>

Re: Questions about HBase

Posted by William Kang <we...@gmail.com>.
Hi Erik and Ryan,
Thanks a lot for your replies.
I think I probably misunderstood something since I saw in HBase
documentation it says "MapFiles cannot currently be mapped into memory". But
the index of it can still be mapped into memory, right? Is it just like
bigtable that the index is at the end of SSTable or else?

And about the row lock question, if I have many users need pull out the same
row (read), what gonna happen?

Thanks.


Cao

On Mon, Mar 1, 2010 at 5:33 PM, Erik Holstad <er...@gmail.com> wrote:

> On Mon, Mar 1, 2010 at 2:16 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
> > Hi,
> >
> > 1.  We use in-memory indexes to get fast random reads.  Our index
> > tells us to read block X of a file only retrieving a small amount of
> > the file to satisfy the user's read.
> >
> > 2.  The row locking is not global - for each row there can only be 1
> > thread doing a put at a time.  This serializes all puts to a single
> > row. It is NOT global.
> >
>
> So, this is me not understanding the difference between global locking and
> waiting in a line until the tasks ahead of you are done :)
>
> >
> > On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>
> > wrote:
> > > Hi guys,
> > > I am new to HBase and have several questions. Would anybody kindly
> answer
> > > some of them?
> > >
> > > 1. Why HBase could provide a low-latency random access to files
> compared
> > to
> > > HDFS?
> > >
> > > 2. By default, Only a single row at a time may be locked. Is it a
> single
> > > client who can only lock one or is it globally can only lock one?  If
> > this
> > > is the case, by default, will the performance be really bad?
> > >
> > > Many thanks!
> > >
> > >
> > > William
> > >
> >
>
>
>
> --
> Regards Erik
>

Re: Questions about HBase

Posted by Ryan Rawson <ry...@gmail.com>.
The ROOT/META are hosted on a regionserver so the system is
self-referential and self-hosting, furthermore it reduces the
dependency on the master to just a single duty of reassigning regions
as necessary.

-ryan

On Mon, Mar 1, 2010 at 7:36 PM, William Kang <we...@gmail.com> wrote:
> And another question, why the ROOT table and META tables are not in the
> HBaseMaster's memory? Would this greatly degrade the performance?
> Thanks.
>
>
> William
>
>
> On Mon, Mar 1, 2010 at 5:33 PM, Erik Holstad <er...@gmail.com> wrote:
>
>> On Mon, Mar 1, 2010 at 2:16 PM, Ryan Rawson <ry...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > 1.  We use in-memory indexes to get fast random reads.  Our index
>> > tells us to read block X of a file only retrieving a small amount of
>> > the file to satisfy the user's read.
>> >
>> > 2.  The row locking is not global - for each row there can only be 1
>> > thread doing a put at a time.  This serializes all puts to a single
>> > row. It is NOT global.
>> >
>>
>> So, this is me not understanding the difference between global locking and
>> waiting in a line until the tasks ahead of you are done :)
>>
>> >
>> > On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>
>> > wrote:
>> > > Hi guys,
>> > > I am new to HBase and have several questions. Would anybody kindly
>> answer
>> > > some of them?
>> > >
>> > > 1. Why HBase could provide a low-latency random access to files
>> compared
>> > to
>> > > HDFS?
>> > >
>> > > 2. By default, Only a single row at a time may be locked. Is it a
>> single
>> > > client who can only lock one or is it globally can only lock one?  If
>> > this
>> > > is the case, by default, will the performance be really bad?
>> > >
>> > > Many thanks!
>> > >
>> > >
>> > > William
>> > >
>> >
>>
>>
>>
>> --
>> Regards Erik
>>
>

Re: Questions about HBase

Posted by William Kang <we...@gmail.com>.
And another question, why the ROOT table and META tables are not in the
HBaseMaster's memory? Would this greatly degrade the performance?
Thanks.


William


On Mon, Mar 1, 2010 at 5:33 PM, Erik Holstad <er...@gmail.com> wrote:

> On Mon, Mar 1, 2010 at 2:16 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
> > Hi,
> >
> > 1.  We use in-memory indexes to get fast random reads.  Our index
> > tells us to read block X of a file only retrieving a small amount of
> > the file to satisfy the user's read.
> >
> > 2.  The row locking is not global - for each row there can only be 1
> > thread doing a put at a time.  This serializes all puts to a single
> > row. It is NOT global.
> >
>
> So, this is me not understanding the difference between global locking and
> waiting in a line until the tasks ahead of you are done :)
>
> >
> > On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>
> > wrote:
> > > Hi guys,
> > > I am new to HBase and have several questions. Would anybody kindly
> answer
> > > some of them?
> > >
> > > 1. Why HBase could provide a low-latency random access to files
> compared
> > to
> > > HDFS?
> > >
> > > 2. By default, Only a single row at a time may be locked. Is it a
> single
> > > client who can only lock one or is it globally can only lock one?  If
> > this
> > > is the case, by default, will the performance be really bad?
> > >
> > > Many thanks!
> > >
> > >
> > > William
> > >
> >
>
>
>
> --
> Regards Erik
>

Re: Questions about HBase

Posted by Ryan Rawson <ry...@gmail.com>.
I guess I interpret 'global locking' as in "affects all threads
regardless of if they are interested in a particular row or not".  Row
locks only affect threads that are interested in said row.  Unrelated
threads are unaffected.  Sorry if there is a little duplication of
words here, it's a fairly subtle yet important issue.



On Mon, Mar 1, 2010 at 2:33 PM, Erik Holstad <er...@gmail.com> wrote:
> On Mon, Mar 1, 2010 at 2:16 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
>> Hi,
>>
>> 1.  We use in-memory indexes to get fast random reads.  Our index
>> tells us to read block X of a file only retrieving a small amount of
>> the file to satisfy the user's read.
>>
>> 2.  The row locking is not global - for each row there can only be 1
>> thread doing a put at a time.  This serializes all puts to a single
>> row. It is NOT global.
>>
>
> So, this is me not understanding the difference between global locking and
> waiting in a line until the tasks ahead of you are done :)
>
>>
>> On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>
>> wrote:
>> > Hi guys,
>> > I am new to HBase and have several questions. Would anybody kindly answer
>> > some of them?
>> >
>> > 1. Why HBase could provide a low-latency random access to files compared
>> to
>> > HDFS?
>> >
>> > 2. By default, Only a single row at a time may be locked. Is it a single
>> > client who can only lock one or is it globally can only lock one?  If
>> this
>> > is the case, by default, will the performance be really bad?
>> >
>> > Many thanks!
>> >
>> >
>> > William
>> >
>>
>
>
>
> --
> Regards Erik
>

Re: Questions about HBase

Posted by Erik Holstad <er...@gmail.com>.
On Mon, Mar 1, 2010 at 2:16 PM, Ryan Rawson <ry...@gmail.com> wrote:

> Hi,
>
> 1.  We use in-memory indexes to get fast random reads.  Our index
> tells us to read block X of a file only retrieving a small amount of
> the file to satisfy the user's read.
>
> 2.  The row locking is not global - for each row there can only be 1
> thread doing a put at a time.  This serializes all puts to a single
> row. It is NOT global.
>

So, this is me not understanding the difference between global locking and
waiting in a line until the tasks ahead of you are done :)

>
> On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com>
> wrote:
> > Hi guys,
> > I am new to HBase and have several questions. Would anybody kindly answer
> > some of them?
> >
> > 1. Why HBase could provide a low-latency random access to files compared
> to
> > HDFS?
> >
> > 2. By default, Only a single row at a time may be locked. Is it a single
> > client who can only lock one or is it globally can only lock one?  If
> this
> > is the case, by default, will the performance be really bad?
> >
> > Many thanks!
> >
> >
> > William
> >
>



-- 
Regards Erik

Re: Questions about HBase

Posted by Ryan Rawson <ry...@gmail.com>.
Hi,

1.  We use in-memory indexes to get fast random reads.  Our index
tells us to read block X of a file only retrieving a small amount of
the file to satisfy the user's read.

2.  The row locking is not global - for each row there can only be 1
thread doing a put at a time.  This serializes all puts to a single
row. It is NOT global.

On Mon, Mar 1, 2010 at 12:36 PM, William Kang <we...@gmail.com> wrote:
> Hi guys,
> I am new to HBase and have several questions. Would anybody kindly answer
> some of them?
>
> 1. Why HBase could provide a low-latency random access to files compared to
> HDFS?
>
> 2. By default, Only a single row at a time may be locked. Is it a single
> client who can only lock one or is it globally can only lock one?  If this
> is the case, by default, will the performance be really bad?
>
> Many thanks!
>
>
> William
>