You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Sumit Agrawal (Jira)" <ji...@apache.org> on 2023/04/25 02:25:00 UTC
[jira] [Updated] (HDDS-8485) SCM Transaction Buffer Delete block delay Investigate solution
[ https://issues.apache.org/jira/browse/HDDS-8485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sumit Agrawal updated HDDS-8485:
--------------------------------
Description:
Investigate the different solution for handling transaction buffer which is causing delete block delay till 1000 ratis transaction is done in SCM,
* SCM flush the db update after 1000 (default) ratis transaction when ratis takes snapshot.
This creates delay, and investicate different mechanism to handlle
3 type of solution as been investigated till now,
# RocksDB BatchWriteWithIndex (HDDS-7483) - This provides similar performance as BatchWrite currently present, and allow to read the data from batchWrite buffer
# Manual WAL flush for rocksDB (HDDS-8469) - This is alternative simpler approach, but current performance comparison is slower, 3-9 times for different scenario
# Timer based solution (HDDS-7111)- Simpler approach flush on time based if not flushed by Ratis (With additional protection of ratis apply transaction parallel - not present currently), Performance will not be hitted
[~erose]
was:
Investigate the different solution for handling transaction buffer which is causing delete block delay till 1000 ratis transaction is done in SCM,
* SCM flush the db update after 1000 (default) ratis transaction when ratis takes snapshot.
This creates delay, and investicate different mechanism to handlle
> SCM Transaction Buffer Delete block delay Investigate solution
> --------------------------------------------------------------
>
> Key: HDDS-8485
> URL: https://issues.apache.org/jira/browse/HDDS-8485
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Sumit Agrawal
> Assignee: Sumit Agrawal
> Priority: Major
>
> Investigate the different solution for handling transaction buffer which is causing delete block delay till 1000 ratis transaction is done in SCM,
> * SCM flush the db update after 1000 (default) ratis transaction when ratis takes snapshot.
> This creates delay, and investicate different mechanism to handlle
>
> 3 type of solution as been investigated till now,
> # RocksDB BatchWriteWithIndex (HDDS-7483) - This provides similar performance as BatchWrite currently present, and allow to read the data from batchWrite buffer
> # Manual WAL flush for rocksDB (HDDS-8469) - This is alternative simpler approach, but current performance comparison is slower, 3-9 times for different scenario
> # Timer based solution (HDDS-7111)- Simpler approach flush on time based if not flushed by Ratis (With additional protection of ratis apply transaction parallel - not present currently), Performance will not be hitted
>
> [~erose]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org