You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Rendon, Carlos (KBB)" <CR...@kbb.com> on 2014/03/28 22:23:56 UTC
Blockcache Eviction on Write?
Hi,
If I have a workload where the access pattern is writing to and reading from the same row, does it make sense to turn the blockcache off? In other words, reads and writes to a given row always go together. Does each write cause eviction of the cached block(s) that the row is stored in? I would think so, but I couldn't find an answer to this anywhere online.
Thanks,
Carlos
Re: Blockcache Eviction on Write?
Posted by Ted Yu <yu...@gmail.com>.
bq. Does each write cause eviction of the cached block(s)
Write first goes to memstore.
If you read the same row immediately after write, the read request would be
served by memstore - if flush hasn't happened.
On Fri, Mar 28, 2014 at 2:23 PM, Rendon, Carlos (KBB) <CR...@kbb.com>wrote:
> Hi,
>
> If I have a workload where the access pattern is writing to and reading
> from the same row, does it make sense to turn the blockcache off? In other
> words, reads and writes to a given row always go together. Does each write
> cause eviction of the cached block(s) that the row is stored in? I would
> think so, but I couldn't find an answer to this anywhere online.
>
> Thanks,
> Carlos
>
Re: Blockcache Eviction on Write?
Posted by Stack <st...@duboce.net>.
On Mon, Mar 31, 2014 at 4:06 PM, Rendon, Carlos (KBB) <CR...@kbb.com>wrote:
> As far as I can tell the cache on write settings are more global than
> just the table in question.
>
> https://hbase.apache.org/book/config.files.html
>
>
>
That is correct. BlockCache is by the server currently. There are no
per-table nor per-cf configs though fellas have been asking for them...
St.Ack
> Is there a table-level option or API level option I’m not aware of?
>
RE: Blockcache Eviction on Write?
Posted by "Rendon, Carlos (KBB)" <CR...@kbb.com>.
As far as I can tell the cache on write settings are more global than just the table in question.
https://hbase.apache.org/book/config.files.html
Is there a table-level option or API level option I’m not aware of?
-Carlos
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Monday, March 31, 2014 2:59 PM
To: Rendon, Carlos (KBB)
Cc: Hbase-User
Subject: Re: Blockcache Eviction on Write?
On Mon, Mar 31, 2014 at 10:37 AM, Rendon, Carlos (KBB) <CR...@kbb.com>> wrote:
Table does not exist yet. I'm expecting random access across the rowkey namespace. I also expect bursts of access to a given row, all of which will change its contents and also read it.
My question is from reading here: https://hbase.apache.org/book/regionserver.arch.html#block.cache
that blockcache wastes memory/CPU/GCs if there aren't many hits to it or if you have random writes that you typically don't read. So I was wondering if writing on every read had the same effect.
After reading the recent thread "Cache invalidation in Blockcache" I can see that during bursts the reads would be coming from the memstore and also that reads still need to be made to the underlying HFiles because data from both sources are merged together to produce a result.
In the scenario I have, it sounds like it is useful to have the HFile blocks cached in the blockcache, even if the data ultimately is returned from the memstore due to a recent write.
And enable cache on write if you are frequently going to be just reading back what you wrote?
St.Ack
Re: Blockcache Eviction on Write?
Posted by Stack <st...@duboce.net>.
On Mon, Mar 31, 2014 at 10:37 AM, Rendon, Carlos (KBB) <CR...@kbb.com>wrote:
> Table does not exist yet. I'm expecting random access across the rowkey
> namespace. I also expect bursts of access to a given row, all of which will
> change its contents and also read it.
>
> My question is from reading here:
> https://hbase.apache.org/book/regionserver.arch.html#block.cache
> that blockcache wastes memory/CPU/GCs if there aren't many hits to it or
> if you have random writes that you typically don't read. So I was wondering
> if writing on every read had the same effect.
>
> After reading the recent thread "Cache invalidation in Blockcache" I can
> see that during bursts the reads would be coming from the memstore and also
> that reads still need to be made to the underlying HFiles because data from
> both sources are merged together to produce a result.
>
> In the scenario I have, it sounds like it is useful to have the HFile
> blocks cached in the blockcache, even if the data ultimately is returned
> from the memstore due to a recent write.
>
And enable cache on write if you are frequently going to be just reading
back what you wrote?
St.Ack
RE: Blockcache Eviction on Write?
Posted by "Rendon, Carlos (KBB)" <CR...@kbb.com>.
Table does not exist yet. I'm expecting random access across the rowkey namespace. I also expect bursts of access to a given row, all of which will change its contents and also read it.
My question is from reading here: https://hbase.apache.org/book/regionserver.arch.html#block.cache
that blockcache wastes memory/CPU/GCs if there aren't many hits to it or if you have random writes that you typically don't read. So I was wondering if writing on every read had the same effect.
After reading the recent thread "Cache invalidation in Blockcache" I can see that during bursts the reads would be coming from the memstore and also that reads still need to be made to the underlying HFiles because data from both sources are merged together to produce a result.
In the scenario I have, it sounds like it is useful to have the HFile blocks cached in the blockcache, even if the data ultimately is returned from the memstore due to a recent write.
Regards,
Carlos
-----Original Message-----
From: Stack [mailto:stack@duboce.net]
Sent: Saturday, March 29, 2014 10:17 AM
To: Hbase-User
Subject: Re: Blockcache Eviction on Write?
What is your blockcache hit rate like? Are your gets totally random across the namespace or is there any locality at all? Can you try turning it off on one of your regionservers and see how it does? Why you want to turn off the block cache, just because it is not being used? There are also these set of configs to try:
http://hbase.apache.org/xref/org/apache/hadoop/hbase/io/hfile/CacheConfig.html#40
If locality of requests, the cache on write might help?
St.Ack
On Fri, Mar 28, 2014 at 2:23 PM, Rendon, Carlos (KBB) <CR...@kbb.com>wrote:
> Hi,
>
> If I have a workload where the access pattern is writing to and
> reading from the same row, does it make sense to turn the blockcache
> off? In other words, reads and writes to a given row always go
> together. Does each write cause eviction of the cached block(s) that
> the row is stored in? I would think so, but I couldn't find an answer to this anywhere online.
>
> Thanks,
> Carlos
>
Re: Blockcache Eviction on Write?
Posted by Stack <st...@duboce.net>.
What is your blockcache hit rate like? Are your gets totally random across
the namespace or is there any locality at all? Can you try turning it off
on one of your regionservers and see how it does? Why you want to turn off
the block cache, just because it is not being used? There are also these
set of configs to try:
http://hbase.apache.org/xref/org/apache/hadoop/hbase/io/hfile/CacheConfig.html#40
If locality of requests, the cache on write might help?
St.Ack
On Fri, Mar 28, 2014 at 2:23 PM, Rendon, Carlos (KBB) <CR...@kbb.com>wrote:
> Hi,
>
> If I have a workload where the access pattern is writing to and reading
> from the same row, does it make sense to turn the blockcache off? In other
> words, reads and writes to a given row always go together. Does each write
> cause eviction of the cached block(s) that the row is stored in? I would
> think so, but I couldn't find an answer to this anywhere online.
>
> Thanks,
> Carlos
>