[jira] [Updated] (HDDS-17112) Avoid decoding BlockData on every single chunk PutBlock with incremental chunk list

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HDDS-17112:
----------------------------------
    Labels: pull-request-available  (was: )

> Avoid decoding BlockData on every single chunk PutBlock with incremental chunk list
> -----------------------------------------------------------------------------------
>
>                 Key: HDDS-17112
>                 URL: https://issues.apache.org/jira/browse/HDDS-17112
>             Project: Apache Ozone
>          Issue Type: Improvement
>            Reporter: Siyao Meng
>            Assignee: Siyao Meng
>            Priority: Major
>              Labels: pull-request-available
>
> With {{ozone.client.incremental.chunk.list=true}} (recommended for HBase), {{DatanodeStoreWithIncrementalChunkList#putBlockWithPartialChunks}} reads and decodes the block's {{BlockData}} from the block data table on every PutBlock. In the most common case, a PutBlock carrying a single partial chunk (for example each hsync of an HBase WAL), the value is only used to check whether the block already exists.
> In a CPU profile of the leader datanode under a WAL like workload (1 KB write followed by hsync, in a loop), this lookup took 3.9% of datanode CPU, and 2.5% of that was decoding the value. The decoding cost grows with the number of full chunks already recorded for the block.
> Proposed fix: use {{Table#isExist}} for the single chunk case, and only get the {{BlockData}} in the multiple chunk branch where it is actually used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org