[jira] [Updated] (HDDS-17100) Do not send chunk checksums in ReadChunk requests

["Siyao Meng (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siyao Meng updated HDDS-17100:
------------------------------
    Status: Patch Available  (was: Open)

> Do not send chunk checksums in ReadChunk requests
> -------------------------------------------------
>
>                 Key: HDDS-17100
>                 URL: https://issues.apache.org/jira/browse/HDDS-17100
>             Project: Apache Ozone
>          Issue Type: Improvement
>          Components: Ozone Client
>            Reporter: Siyao Meng
>            Assignee: Siyao Meng
>            Priority: Major
>              Labels: pull-request-available
>
> {{ChunkInputStream#getChunkInfo}} builds the {{ChunkInfo}} of every ReadChunk request by copying the chunk's full {{ChunkInfo}}, including the checksum of every {{bytesPerChecksum}} slice of the chunk (256 checksums for a 4 MB chunk at 16 KB per checksum) and, for EC, the stripe checksum. This happens even for a small positioned read.
> The datanode does not use these checksums for client reads (it only verifies data read for Ratis, see HDDS-5623), but it parses them and echoes the request {{ChunkInfo}} back in the response, which the client parses again. The client verifies the data against its own copy of the chunk checksums, not the echoed ones.
> On a 3 datanode docker compose cluster with random 64 KB preads from 8 threads, parsing the ReadChunk request was about 12% of datanode CPU, almost all of it in {{ChecksumData$Builder.mergeFrom}}. Leaving the checksums out of the request cut datanode CPU per read from about 58 µs to 34 µs (request parsing dropped to 2.6% of datanode CPU). Latency on the idle test cluster was unchanged within noise.
> Proposed fix: clear {{checksums}} (keeping type and {{bytesPerChecksum}}) and {{stripeChecksum}} in the ReadChunk request {{ChunkInfo}}. The change is client side only and works with existing datanodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org