[jira] [Updated] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream

["Yi Liu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yi Liu updated HDFS-8223:
-------------------------
    Attachment: HDFS-8223.001.patch

Upload the patch.

The idea is to calculate the checksums for a parity cell once, and then send them in the packets.

We don't need to add new tests, existing {{TestDFSStripedOutputStream}} can cover the testing.  Since if we calculate checksum for the striped blocks and write into the packet, then DataNode will check it, I have verified this through debug too.

I don't use {{byteArrayManager}} to allocate bytes buffer for {{checksumBuf}}, since the buffer is small and only one in one stream instance.

> Should calculate checksum for parity blocks in DFSStripedOutputStream
> ---------------------------------------------------------------------
>
>                 Key: HDFS-8223
>                 URL: https://issues.apache.org/jira/browse/HDFS-8223
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-8223.001.patch
>
>
> {quote}
> DFSPacket p = createPacket(packetSize, chunksPerPacket,
>           streamer.getBytesCurBlock(),
>           streamer.getAndIncCurrentSeqno(), false);
>       int maxBytesToPacket = p.getMaxChunks() * bytesPerChecksum;
>       int toWrite = byteBuffer.remaining() > maxBytesToPacket ?
>           maxBytesToPacket: byteBuffer.remaining();
>       p.writeData(byteBuffer, toWrite);
>       streamer.incBytesCurBlock(toWrite);
>       packets.add(p);
> {quote}
> In {{DFSStripedOutputStream}}, have not calculated checksum for parity blocks in {{DFSStripedOutputStream}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)