You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/03/30 04:24:00 UTC
[jira] [Updated] (HDDS-14921) Improve space accounting in SCM with In-Flight container allocation tracking
[ https://issues.apache.org/jira/browse/HDDS-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-14921:
----------------------------------
Labels: pull-request-available (was: )
> Improve space accounting in SCM with In-Flight container allocation tracking
> ----------------------------------------------------------------------------
>
> Key: HDDS-14921
> URL: https://issues.apache.org/jira/browse/HDDS-14921
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Ashish Kumar
> Assignee: Ashish Kumar
> Priority: Major
> Labels: pull-request-available
>
> The current disk space management and container allocation mechanism in SCM (Storage Container Manager) relies heavily on periodic DataNode (DN) heartbeat reports and static policies such as pipeline count based on disk numbers. This approach introduces multiple systemic challenges:
> # *Stale Space Visibility*
> SCM makes allocation decisions based on heartbeat-reported disk space, which can lag behind actual usage. During high allocation rates, this delay leads to inaccurate space estimation and potential over-allocation.
> # *Burst Allocation Risk*
> Rapid container allocations within short intervals are not accounted for immediately, allowing multiple allocations against the same reported free space. This can oversubscribe disks and result in sudden disk exhaustion.
> *Solution:*
> Two Window Tumbling Bucket similar like HADOOP-3707
> *Two Windows Per DataNode*
> Each DataNode has a TwoWindowBucket containing:
> - currentWindow: Containers allocated in the current 10-minute interval
> - previousWindow: Containers from the previous 10-minute interval
> - lastRollTime: Timestamp of last roll
> *Container Allocation Flow: New Container Allocated*
> - Add ContainerID to currentWindow
> - Check if roll needed (time > lastRollTime + 10min)
> - If yes: previousWindow = currentWindow; currentWindow = {}
> *Space check: Get Pending Allocations*
> - Roll if needed
> - Return UNION(currentWindow, previousWindow)
> - pendingSize = union.size() × maxContainerSize
> - effectiveSpace = remainingSpace - pendingSize
> *Container report: Container Report Received*
> - Remove ContainerID from BOTH windows
> - More accurate than waiting for automatic aging
> - Falls back to aging if report is delayed/missed **
> *Automatic Aging: Roll*
> Every 10 Minutes (Triggered Lazily on Operations):
> 1. previousWindow = currentWindow
> 2. currentWindow = {} (new empty set)
> 3. lastRollTime = now
> 4. Old previousWindow is garbage collected
>
> +*Timeline Example*+
> Time | Action | CurrentWindow | PreviousWindow | Total Pending
> ------+---------------------------+---------------+----------------+--------------
> 00:00 | Allocate Container-1 | \{C1} | {} | \{C1}
> 00:05 | Allocate Container-2 | \{C1, C2} | {} | \{C1, C2}
> 00:08 | Allocate Container-3 | \{C1, C2, C3} | {} | \{C1, C2, C3}
> 00:10 | [ROLL] Window tumbles | {} | \{C1, C2, C3} | \{C1, C2, C3}
> | ⤷ previousWindow ← currentWindow
> | ⤷ currentWindow ← {} (reset)
> ------+---------------------------+---------------+----------------+--------------
> 00:12 | Allocate Container-4 | \{C4} | \{C1, C2, C3} | \{C1, C2, C3, C4}
> 00:15 | Report confirms C1 | \{C4} | \{C2, C3} | \{C2, C3, C4}
> | ⤷ Explicitly removed from previousWindow
> 00:18 | Allocate Container-5 | \{C4, C5} | \{C2, C3} | \{C2, C3, C4, C5}
> 00:20 | [ROLL] Window tumbles | {} | \{C4, C5} | \{C4, C5}
> | ⤷ C2, C3 aged out (not reported in 20 min)
> ------+---------------------------+---------------+----------------+--------------
> 00:25 | Report confirms C4 | {} | \{C5} | \{C5}
> 00:30 | [ROLL] Window tumbles | {} | {} | {}
> | ⤷ C5 aged out (not reported in 20 min)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org