You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Prashant Pogde (Jira)" <ji...@apache.org> on 2021/01/29 17:23:14 UTC

[jira] [Updated] (HDDS-2309) Optimise OzoneManagerDoubleBuffer::flushTransactions to flush in batches

     [ https://issues.apache.org/jira/browse/HDDS-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prashant Pogde updated HDDS-2309:
---------------------------------
    Target Version/s: 1.2.0

I am managing the 1.1.0 release and we currently have more than 600 issues targeted for 1.1.0. I am moving the target field to 1.2.0. 

If you are actively working on this jira and believe this should be targeted to 1.1.0 release, Please change the target field back to 1.1.0 before Feb 05, 2021. 

> Optimise OzoneManagerDoubleBuffer::flushTransactions to flush in batches
> ------------------------------------------------------------------------
>
>                 Key: HDDS-2309
>                 URL: https://issues.apache.org/jira/browse/HDDS-2309
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Manager
>            Reporter: Rajesh Balamohan
>            Assignee: Bharat Viswanadham
>            Priority: Major
>              Labels: TriagePending, performance
>         Attachments: Screenshot 2019-10-15 at 4.19.13 PM.png
>
>
> When running a write heavy benchmark, {{{color:#000000}org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.flushTransactions{color}}} was invoked for pretty much every write.
> This forces {{cleanupCache}} to be invoked which ends up choking in single thread executor. Attaching the profiler information which gives more details.
> Ideally, {{flushTransactions}} should batch up the work to reduce load on rocksDB.
>  
> [https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L130]
>  
> [https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L322]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org