You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Sever Fundatureanu <fu...@gmail.com> on 2012/06/27 13:18:31 UTC
In memory table after using 'alter'
Hello,
I initially created a table without the IN_MEMORY option enabled and loaded
some data into it. Then I disabled it, modified the IN_MEMORY option using
the hbase shell 'alter' command, re-enabled it and finally ran a major
compaction. I do notice now the memory usage of the region servers has
increased. However for some queries I am still getting faster responses
only the 2nd time, as if I'm hitting the cache. Can someone tell me when
does the table content get transferred into RAM if IN_MEMORY option is
enabled like above?
Thank you,
--
Sever Fundatureanu
Vrije Universiteit Amsterdam
E-mail: fundatureanu.sever@gmail.com
Re: In memory table after using 'alter'
Posted by Minh Duc Nguyen <md...@gmail.com>.
Sever, I'd say that the wiki page is incomplete when discussing bloom
filters. The HBase Reference Guide
http://hbase.apache.org/book.html#schema.bloom
provides a much better explanation of the different bloom filter options.
~ Minh
On Thu, Jun 28, 2012 at 9:54 AM, Sever Fundatureanu
<fu...@gmail.com> wrote:
> Thanks, Minh.
> This blog also mentions that bloom filters check "if a given column exists
> in a given row". However in "HBase: The definitive guide" it is mentioned
> that "enabling the Bloom filter does give you the immediate advantage of
> knowing if a file contains a particular row key or not". Is the blog post a
> bit outdated?
>
> Thanks,
> Sever
>
>
> On Wed, Jun 27, 2012 at 6:00 PM, Minh Duc Nguyen <md...@gmail.com> wrote:
>
>> Sever, the IN_MEMORY option doesn't change when table content is
>> transferred into RAM. Whether set to true or false, the blocks of
>> data are only loaded into memory after a normal retrieval operation.
>> When IN_MEMORY is set to true, HBase just tries to keep data in memory
>> more aggressively than it normally would. This wiki explains it in
>> more detail:
>>
>>
>> http://jimbojw.com/wiki/index.php?title=Understanding_HBase_column-family_performance_options
>>
>> ~ Minh
>>
>> On Wed, Jun 27, 2012 at 7:18 AM, Sever Fundatureanu
>> <fu...@gmail.com> wrote:
>> > Hello,
>> >
>> > I initially created a table without the IN_MEMORY option enabled and
>> loaded
>> > some data into it. Then I disabled it, modified the IN_MEMORY option
>> using
>> > the hbase shell 'alter' command, re-enabled it and finally ran a major
>> > compaction. I do notice now the memory usage of the region servers has
>> > increased. However for some queries I am still getting faster responses
>> > only the 2nd time, as if I'm hitting the cache. Can someone tell me when
>> > does the table content get transferred into RAM if IN_MEMORY option is
>> > enabled like above?
>> >
>> > Thank you,
>> > --
>> > Sever Fundatureanu
>> >
>> > Vrije Universiteit Amsterdam
>> > E-mail: fundatureanu.sever@gmail.com
>>
>
>
>
> --
> Sever Fundatureanu
>
> Vrije Universiteit Amsterdam
> Tel: +31644329534
> E-mail: fundatureanu.sever@gmail.com
Re: In memory table after using 'alter'
Posted by Sever Fundatureanu <fu...@gmail.com>.
Thanks, Minh.
This blog also mentions that bloom filters check "if a given column exists
in a given row". However in "HBase: The definitive guide" it is mentioned
that "enabling the Bloom filter does give you the immediate advantage of
knowing if a file contains a particular row key or not". Is the blog post a
bit outdated?
Thanks,
Sever
On Wed, Jun 27, 2012 at 6:00 PM, Minh Duc Nguyen <md...@gmail.com> wrote:
> Sever, the IN_MEMORY option doesn't change when table content is
> transferred into RAM. Whether set to true or false, the blocks of
> data are only loaded into memory after a normal retrieval operation.
> When IN_MEMORY is set to true, HBase just tries to keep data in memory
> more aggressively than it normally would. This wiki explains it in
> more detail:
>
>
> http://jimbojw.com/wiki/index.php?title=Understanding_HBase_column-family_performance_options
>
> ~ Minh
>
> On Wed, Jun 27, 2012 at 7:18 AM, Sever Fundatureanu
> <fu...@gmail.com> wrote:
> > Hello,
> >
> > I initially created a table without the IN_MEMORY option enabled and
> loaded
> > some data into it. Then I disabled it, modified the IN_MEMORY option
> using
> > the hbase shell 'alter' command, re-enabled it and finally ran a major
> > compaction. I do notice now the memory usage of the region servers has
> > increased. However for some queries I am still getting faster responses
> > only the 2nd time, as if I'm hitting the cache. Can someone tell me when
> > does the table content get transferred into RAM if IN_MEMORY option is
> > enabled like above?
> >
> > Thank you,
> > --
> > Sever Fundatureanu
> >
> > Vrije Universiteit Amsterdam
> > E-mail: fundatureanu.sever@gmail.com
>
--
Sever Fundatureanu
Vrije Universiteit Amsterdam
Tel: +31644329534
E-mail: fundatureanu.sever@gmail.com
Re: In memory table after using 'alter'
Posted by Minh Duc Nguyen <md...@gmail.com>.
Sever, the IN_MEMORY option doesn't change when table content is
transferred into RAM. Whether set to true or false, the blocks of
data are only loaded into memory after a normal retrieval operation.
When IN_MEMORY is set to true, HBase just tries to keep data in memory
more aggressively than it normally would. This wiki explains it in
more detail:
http://jimbojw.com/wiki/index.php?title=Understanding_HBase_column-family_performance_options
~ Minh
On Wed, Jun 27, 2012 at 7:18 AM, Sever Fundatureanu
<fu...@gmail.com> wrote:
> Hello,
>
> I initially created a table without the IN_MEMORY option enabled and loaded
> some data into it. Then I disabled it, modified the IN_MEMORY option using
> the hbase shell 'alter' command, re-enabled it and finally ran a major
> compaction. I do notice now the memory usage of the region servers has
> increased. However for some queries I am still getting faster responses
> only the 2nd time, as if I'm hitting the cache. Can someone tell me when
> does the table content get transferred into RAM if IN_MEMORY option is
> enabled like above?
>
> Thank you,
> --
> Sever Fundatureanu
>
> Vrije Universiteit Amsterdam
> E-mail: fundatureanu.sever@gmail.com