You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Rajesh Balamohan (Jira)" <ji...@apache.org> on 2019/10/23 13:29:00 UTC
[jira] [Created] (HDDS-2351) Fix write performance issue in Non-HA
OM
Rajesh Balamohan created HDDS-2351:
--------------------------------------
Summary: Fix write performance issue in Non-HA OM
Key: HDDS-2351
URL: https://issues.apache.org/jira/browse/HDDS-2351
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: om
Reporter: Rajesh Balamohan
Attachments: Screenshot 2019-10-23 at 2.27.05 PM.png
HDDS-2333 enables sync option in OM non-HA mode. However, this flushes very frequently causing disk to saturate its IOPS soon. It creates way too small write workloads and disk hits limit.
To put it in perspective, in simple write benchmark of creating keys with 10 clients, it generates {{0.33MB/s}} write workload with {{116 IOPS}}. This causes disk to saturate at 98%.
!Screenshot 2019-10-23 at 2.27.05 PM.png|width=621,height=370!
Reverting back HDDS-2333 fixes this issue. I see {{10x}} degradation with HDDS-2333.
In case non-HA is supported in OM, it would be good to call it out. Currently, code explicitly enables sync option. [https://github.com/apache/hadoop-ozone/commit/c6c9794fc590371ad9c3b8fdcd7a36ed42909b40#diff-3ed3ab4891d7b4fa31ca96740b78ae5bR261]
I used {{commit 1baa5a158d13f469c12bef86ef288d60ef0eee85}} in master branch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org