[jira] [Resolved] (HDDS-4459) Improve closed container replication

["Marton Elek (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marton Elek resolved HDDS-4459.
-------------------------------
    Resolution: Fixed

Resolving it for now as all the subtasks are resolved. We need further improvement on closed-container replication but we can create either new "phase2" parent jira or repone this one.

> Improve closed container replication
> ------------------------------------
>
>                 Key: HDDS-4459
>                 URL: https://issues.apache.org/jira/browse/HDDS-4459
>             Project: Apache Ozone
>          Issue Type: Improvement
>            Reporter: Marton Elek
>            Assignee: Marton Elek
>            Priority: Major
>
> Closed are immutable therefore the replication is different from Ratis.
> Closed containers are downloaded from other datanodes over a specific GRPC channel.
> While the first implementation works as intended it has many spaces for improvements.
>  1. the container replica is compressed with gzip. A more efficient compression codec like zstd might be better (reported Wei-Chiu)
>  2. Current downloader uses only one thread (reported Wei-Chiu)
>  3. Temporary space is configurable, but by default uses /tmp. This means we are bottlenecked by the root disk, even though we have 48 disks on each DN. There should be an option to store the temp file under the destination disk instead. (reported Wei-Chiu)
>  4. Buffer size is hard coded (1MB) and may not be the ideal
>  5. There are no metrics which clearly shows the performance of the replication
>  6. Separate client and server2server RPC listeners
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org