[jira] [Created] (HDDS-9458) Throttling strategy for write operation in container state machine

["Sumit Agrawal (Jira)" <ji...@apache.org>]

Sumit Agrawal created HDDS-9458:
-----------------------------------

             Summary: Throttling strategy for write operation in container state machine
                 Key: HDDS-9458
                 URL: https://issues.apache.org/jira/browse/HDDS-9458
             Project: Apache Ozone
          Issue Type: Sub-task
            Reporter: Sumit Agrawal
            Assignee: Sumit Agrawal


h3. Throttling Strategy:  
 * *Request count*

If pending request > 500 (50%), *5 request/Sec*

If pending request > 800 (80%), *1 request/Sec*

If pending request > 1000 (95%), reject as ServerBusy

 

Need block only which involves data:
 * putSmallFile
 * WriteChunk required

 
 # *Request Size*

If pending request size > 500MB (50%), *5 request/Sec*

If pending request size > 800MB (80%), *1 request/Sec*

If pending request size > 1000MB (99%), reject as ServerBusy

 

Need block only which involves data:
 * putSmallFile
 * WriteChunk required

 
 * *System memory usages (-XMX) - Global memory control*

 If Overall pending request size > 70% of Max System memory, *5 request/Sec*

 If Overall pending request size > 85% of Max System memory, reject as Server Busy

 

Need block only which involves data:
 * putSmallFile
 * WriteChunk required

 

*Note:* If incoming request rate is high causing accumulation of request > 1024 or 1GB size (as default configuration) in throttling queue, start rejecting further requests. Also need an Error log in the health checker for the same.

 
h3. Configuration for throttling:
 * Rate of request processing for request count and size

dfs.container.throtelling.request.count=50,5,70,1,95,0

dfs.container.throtelling.request.size=50,5,70,1,95,0

 
 * Rate of request processing for system memory usages

dfs.container.throtelling.memory.request.size=70,5,85,0

 
 * Enable throttling (default enable)

dfs.container.throtelling.enable=true



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org