Calculation of packet size in DFSOutputStream

[Verdi March <ve...@gmail.com>]

Hi,

I'd like to ask about the calculation of packet size, as implemented in
DFSClient.DFSOutputStream (Hadoop v1.0.3). My apology if this is not
the appropriate list.

1.  private void computePacketChunkSize(int psize, int csize) {
2.    int chunkSize = csize + checksum.getChecksumSize();
3.    int n = DataNode.PKT_HEADER_LEN + SIZE_OF_INTEGER;
4.    chunksPerPacket = Math.max((psize - n + chunkSize-1)/chunkSize, 1);
5.    packetSize = n + chunkSize*chunksPerPacket;
6.    if (LOG.isDebugEnabled()) {
7.      LOG.debug("computePacketChunkSize: src=" + src +
8.                ", chunkSize=" + chunkSize +
9.                ", chunksPerPacket=" + chunksPerPacket +
10.               ", packetSize=" + packetSize);
11.   }
12. }

My question is re. line-4. The debug log shows that the packet header
(PKT_HEADER_LEN) is not part of the payload (64KB by default), so why
still substract psize with n instead of with SIZE_OF_INTEGER?

I'm not entirely familiar with the intricacies of HDFS, so am hoping
for some pointers or directions.

Regards,
Verdi