How can i disable filesystem cache

[Gui Zhang <al...@gmail.com>]

I want test hbase cache perfermance, as i know linux alway try to cache the
file, so i would like to disable filesystem cache.
my hbase is build on top of hadoop, OS is redhat linux。

So here i actually need disable fs cache for hadoop files.
Is this possible and how?

Thanks
Gui

Re: How can i disable filesystem cache

[lars hofhansl <lh...@yahoo.com>]

Hi Gui,

you cannot disable the filesystem cache. Linux always reads through the page/buffer cache.


If this is for testing non-cached performance, you can empty cache by writing to /proc/sys.vm/drop_caches...


To free pagecache (as root):

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

-- Lars



________________________________
From: Gui Zhang <al...@gmail.com>
To: user@hbase.apache.org
Sent: Thursday, August 25, 2011 12:49 AM
Subject: How can i disable filesystem cache

I want test hbase cache perfermance, as i know linux alway try to cache the
file, so i would like to disable filesystem cache.
my hbase is build on top of hadoop, OS is redhat linux。

So here i actually need disable fs cache for hadoop files.
Is this possible and how?

Thanks
Gui

Re: How can i disable filesystem cache

[Edward Capriolo <ed...@gmail.com>]

On Thu, Aug 25, 2011 at 1:35 PM, Li Pi <li...@cloudera.com> wrote:

> You could just use a test size that always fits within the hbase cache.
> On Aug 25, 2011 12:50 AM, "Gui Zhang" <al...@gmail.com> wrote:
> > I want test hbase cache perfermance, as i know linux alway try to cache
> the
> > file, so i would like to disable filesystem cache.
> > my hbase is build on top of hadoop, OS is redhat linux。
> >
> > So here i actually need disable fs cache for hadoop files.
> > Is this possible and how?
> >
> > Thanks
> > Gui
>

For a system with 16GB RAM you can set your -Xms and Xmx to a very high
number like 14 of 15GB. Or run some other process that hogs all your system
memory, like write a C+program that allocates huge a really large linked
list and then sleeps. You really do not want to take away ALL of your VFS
file cache it gets very painful when your inode table is not cached or the
libraries that bash uses are not cached.

Re: How can i disable filesystem cache

[Li Pi <li...@cloudera.com>]

You could just use a test size that always fits within the hbase cache.
On Aug 25, 2011 12:50 AM, "Gui Zhang" <al...@gmail.com> wrote:
> I want test hbase cache perfermance, as i know linux alway try to cache
the
> file, so i would like to disable filesystem cache.
> my hbase is build on top of hadoop, OS is redhat linux。
>
> So here i actually need disable fs cache for hadoop files.
> Is this possible and how?
>
> Thanks
> Gui