Possible issue with SequenceFileBolt

[Sachin Pasalkar <Sa...@symantec.com>]

When you use the compressionType as BLOCK it does not rotate the file at specified file size.

On further investigation I found that Block writer sync only when buffered data reached compressionBlockSize.
Code in org.apache.hadoop.io.SequenceFile.BlockCompressWriter.append(Object, Object)

 int currentBlockSize = keyBuffer.getLength() + valBuffer.getLength();
            if(currentBlockSize >= compressionBlockSize)
                sync();
Default value of compressionBlockSize is 1000000, which make
offset = this.writer.getLength(); (SequenceFileBolt class) to returns stall value of current data size.

Incase or org.apache.hadoop.io.SequenceFile.RecordCompressWriter it works fine as it always call for sync api (checkAndWriteSync())

Either we can sync records every time it comes or use  int currentBlockSize = keyBuffer.getLength() + valBuffer.getLength(); to return current size.

Thanks,
Sachin