too many disk IOs / chunkserver suddenly consumes ~100GB of address space

[Andrew Purtell <ap...@apache.org>]

Hi,

Like Ryan I have been trying to run HBase on top of
KFS. In my case I am running a SVN snapshot from 
yesterday. I have a minimal installation of KFS
metaserver, chunkserver, and HBase master and
regionserver all running on one test host with 4GB of
RAM. Of course I do not expect more than minimal
function. To apply some light load, I run the Heritrix
crawler with 5 TOE threads which write on average
200 Kbit/sec of data into HBase, which flushes this
incoming data in ~64MB increments and also runs
occasional compaction cycles where the 64MB flush
files will be compacted into ~256MB files. 

I find that for no obvious reason the chunkserver will
suddenly grab ~100 GIGAbytes of address space and emit
a steady stream of "(DiskManager.cc:392) Too many disk
IOs (N)" to the log at INFO level, where N is a
steadily increasing number. The host is under moderate
load at the time -- KFS is busy -- but is not in swap
and according to atop has some disk I/O and network
bandwidth to spare. 

Any ideas as to what might be the cause? Is there more
information I can provide to help diagnose the problem?

Best regards,

   - Andy