You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Kirill Sizov (Jira)" <ji...@apache.org> on 2023/10/06 10:46:00 UTC
[jira] [Updated] (HDDS-9404) SCM executes all pipeline operations under single lock
[ https://issues.apache.org/jira/browse/HDDS-9404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kirill Sizov updated HDDS-9404:
-------------------------------
Description:
This task came out of the performance testing.
*Cluster Setup*
12 nodes: 3 master, each running SCM and OM and 9 data nodes, each running DN and S3G.
2 CPUs x 24 cores each, having totally 96 vCPUs.
Each node has 12 SSDs. And 256G RAM.
*Load generators*
Running from 3 nodes, each spawned 25 threads.
Total load - uploading 100k files, each file is 20MB.
*Ozone Setup*
The writes were putting files into a single Erasure Coded bucket RS-6-3-1024k.
*Observations*
We have seen periodic drops in the FPS to almost zero.
was:
This task came out of the performance testing.
*Cluster Setup*
12 nodes: 3 master, each running SCM and OM and 9 data nodes, each running DN and S3G.
2 CPUs x 24 cores each, having totally 96 vCPUs.
Each node has 12 SSDs. And 256G RAM.
*Load generators*
Running from 3 nodes, each spawned 25 threads.
Total load - uploading 100k files, each file is 20MB.
*Ozone Setup*
The writes were putting files into a single Erasure Coded bucket RS-6-3-1024k.
*Observations*
We have seen periodic drops in the FPS to almost zero.
> SCM executes all pipeline operations under single lock
> ------------------------------------------------------
>
> Key: HDDS-9404
> URL: https://issues.apache.org/jira/browse/HDDS-9404
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Kirill Sizov
> Priority: Major
> Attachments: load.png
>
>
> This task came out of the performance testing.
> *Cluster Setup*
> 12 nodes: 3 master, each running SCM and OM and 9 data nodes, each running DN and S3G.
> 2 CPUs x 24 cores each, having totally 96 vCPUs.
> Each node has 12 SSDs. And 256G RAM.
> *Load generators*
> Running from 3 nodes, each spawned 25 threads.
> Total load - uploading 100k files, each file is 20MB.
> *Ozone Setup*
> The writes were putting files into a single Erasure Coded bucket RS-6-3-1024k.
> *Observations*
> We have seen periodic drops in the FPS to almost zero.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org