[jira] [Assigned] (HDDS-1094) Performance testing infrastructure : Special handling for zero-filled chunks on the Datanode

["Supratim Deka (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Supratim Deka reassigned HDDS-1094:
-----------------------------------

    Assignee: Supratim Deka

> Performance testing infrastructure : Special handling for zero-filled chunks on the Datanode
> --------------------------------------------------------------------------------------------
>
>                 Key: HDDS-1094
>                 URL: https://issues.apache.org/jira/browse/HDDS-1094
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>          Components: Ozone Datanode
>            Reporter: Supratim Deka
>            Assignee: Supratim Deka
>            Priority: Major
>
> Goal:
> Make Ozone chunk Read/Write operations CPU/network bound for specially constructed performance micro benchmarks.
> Remove disk bandwidth and latency constraints - running ozone data path against extreme low-latency & high throughput storage will expose performance bottlenecks in the flow. But low-latency storage(NVME flash drives, Storage class memory etc) is expensive and availability is limited. Is there a workaround which achieves similar running conditions for the software without actually having the low latency storage? At least for specially constructed datasets -  for example zero-filled blocks (*not* zero-length blocks).
> Required characteristics of the solution:
> No changes in Ozone client, OM and SCM. Changes limited to Datanode, Minimal footprint in datanode code.
> Possible High level Approach:
> The ChunkManager and ChunkUtils can enable writeChunk for zero-filled chunks to be dropped without actually writing to the local filesystem. Similarly, if readChunk can construct a zero-filled buffer without reading from the local filesystem whenever it detects a zero-filled chunk. Specifics of how to detect and record a zero-filled chunk can be discussed on this jira. Also discuss how to control this behaviour and make it available only for internal testing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org