a problem with a single large file in hadoop

[Da Zheng <zh...@gmail.com>]

Hello,

I have a file of 25GB and I put it in the Hadoop file system. After I ran my
MapReduce program on it, I found the input size shown in "Map input bytes" is
only about 12GB. So I tried different sizes of a file, and found that as long as
the file size is larger than 4GB, the size shown in "Map input bytes" is smaller
than the input file. I also checked ganglia, and the bytes read in data nodes
are the same as the size shown in "Map input bytes". Has anyone got this problem
before? Is it a bug of Hadoop or the user are supposed to split large files
before putting it to the Hadoop file system?

Best,
Da