local bytes written (high io, low memory usage)

[Haijun Cao <ha...@kindsight.net>]

I noticed that "local bytes written/read" stat in my map reduce job is
really high, 2x, 3x, 4x of the hdfs bytes.  

 

When does hadoop mapred framework write to local fs? Is it done when the
jvm memory is not enough and data is spill to disk? how I can configure
so that it does not spill to disk? My memory usage is really low, while
io is nearly saturated.

 

Thanks

Haijun