Buffering to minimize Disk IO

[david guimbellot <dg...@hotmail.com>]

All, 
 
We use log4j across a number of services where many servers write to a shared disk. The net impact of the log writes contributes to the total number of physical disk writes that the disk can support. This has become our overall system bottleneck (NOT THE CPU)
 
Has anyone tried tuning the logging system to the behavior of the underlying disk layout?
 
__Rough Algorithm Suggestion__
Create a buffer that is aligned with the optimal write size of the physical disk medium. You can find this out for your system via IOmeter; http://sourceforge.net/projects/iometer.
 
Append the random sized log messages into the aligned buffer and only flush to disk the buffer fills.
 
This would reduce the number of physical disk writes to the underlying medium and thus improve the overall system performance.
 
Thanks
dg