You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by elton sky <el...@gmail.com> on 2010/07/28 12:36:15 UTC

io.file.buffer.size, how hadoop uses it?

I am a bit confused of how this attribute is used.

My understanding is it's related with file read/write. And I can see, in
LineReader.java, it's used as the default buffer size for each line; in
BlockReader.newBlockReader(), it's used as the internal buffer size of the
BufferedInputStream. Also, in compression related classes, it's used as
default buffer size. However, when creating a file (write), bufferSize is
not seemed to be used at all.

E.g.
DFSClient.DFSOutputStream(String src, int buffersize, Progressable progress,
LocatedBlock lastBlock, FileStatus stat,int bytesPerChecksum);
it has a buffersize param, but never used in its definition. In other words,
it's not used for writing at all?

Is this right?