You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "0lai0 (via GitHub)" <gi...@apache.org> on 2026/01/22 09:51:23 UTC
[PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
0lai0 opened a new pull request, #276:
URL: https://github.com/apache/ozone-site/pull/276
## What changes were proposed in this pull request?
Publish a blog to give a preview of a new feature to be included in Ozone 2.2: disk balancer.
Blog post draft: https://docs.google.com/document/d/1hxOZYZ97wNgzbWb0dygVvjUC-cC3RnahB-C6GRn8Y60/edit?tab=t.0#heading=h.rxs6rcvswuwy
**Screenshot**
<img width="1920" height="962" alt="image" src="https://github.com/user-attachments/assets/aadaf123-c8d3-46df-8e29-3f056140a4d0" />
## What is the link to the Apache Jira?
[https://issues.apache.org/jira/browse/HDDS-14324](https://issues.apache.org/jira/browse/HDDS-14324)
## How was this patch tested?
Check off which of the following tests were done on this change. If additional testing was done, please elaborate here as well.
- [x] The CI checks on my fork are passing
- [x] I verified the rendered content using a local preview
- [x] I manually verified the steps provided in this change work as described
GitHub Actions CI: [https://github.com/0lai0/ozone-site/actions/runs/21243667131](https://github.com/0lai0/ozone-site/actions/runs/21243667131)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324. [Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3816381852
Merged. Thanks all!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2733442649
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
Review Comment:
update this when we publish
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+**Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode; SCM just receives reports.
+
+## Using Disk Balancer
Review Comment:
Foremost, enable the disk balancer feature by setting hdds.datanode.disk.balancer.enabled = true in ozone-site.xml on Datanodes.
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
Review Comment:
@0lai0 @Gargi-jais11 would you like to add your as the authors? We can add the author info in the blog/authors.yaml.
```
apache-ozone-community:
name: The Apache Ozone Community
title: Apache Ozone Project
url: https://ozone.apache.org
image_url: /img/ozone-logo.svg
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324. [Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #276:
URL: https://github.com/apache/ozone-site/pull/276
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3807081238
Tagging Shopee team @ferhui @symious @JacksonYao287 FYI.
I'd like to hash it out and have it published later this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739920529
##########
blog/2026-01-29-disk-balancer-preview.md:
##########
@@ -0,0 +1,103 @@
+---
+title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
+authors: ["apache-ozone-community","jojochuang", "0lai0","Gargi-jais11"]
+date: 2026-01-29
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+Ever replaced a drive on a Datanode only to watch it become an I/O hotspot?
+Or seen one disk hit 95% usage while others on the same machine sit idle?
+These imbalances create performance bottlenecks and increase failure risk.
+Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode. To use it, first enable the feature by setting `hdds.datanode.disk.balancer.enabled = true` in `ozone-site.xml` on your Datanodes. Once enabled, clients use `ozone admin datanode diskbalancer` commands to talk directly to Datanodes, with SCM only used to discover IN_SERVICE datanodes when running batch operations with `--in-service-datanodes`.
+
+### Disk Balancing Policies
+
+DiskBalancer uses simple but robust policies to decide **which disks to balance** and **which containers to move** (see the design doc for details: `diskbalancer.md` in [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)).
+
+- **Default Volume Choosing Policy**: Picks the most over‑utilized volume as the source and the most under‑utilized volume as the destination, based on each disk’s **Volume Data Density** and the Datanode’s average utilization.
+- **Default Container Choosing Policy**: Scans containers on the source volume and moves only **CLOSED** containers that are not already being moved. To avoid repeatedly scanning the same list, it caches container metadata with automatic expiry.
+
+These defaults aim to make safe, incremental moves that converge the disks toward an even utilization state.
+
+### Container Move Process
+
+When DiskBalancer moves a container from one disk to another on the **same Datanode**, it follows a careful copy‑and‑import flow (summarized from the design doc for [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)):
+
+1. Create a temporary copy of the CLOSED container on the destination disk.
+2. Transition that copy into a **RECOVERING** state and import it as a new container on the destination.
+3. Once import and metadata updates succeed, delete the original CLOSED container from the source disk.
+
+This ensures that data is always consistent: the destination copy is fully validated before the original is removed, minimizing risk during balancing.
+
+## Using Disk Balancer
+
+First, enable the Disk Balancer feature on each Datanode by setting the following in `ozone-site.xml`:
+
+- `hdds.datanode.disk.balancer.enabled = true`
+
+CLI examples:
+
+```bash
+# Start DiskBalancer on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer start --in-service-datanodes
+
+# Stop DiskBalancer on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer stop --in-service-datanodes
+
+# Check DiskBalancer status on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer status --in-service-datanodes
+
+# Get a volume density report to see imbalance across disks
+ozone admin datanode diskbalancer report --in-service-datanodes
+
+# Update DiskBalancer configuration on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer update --in-service-datanodes
+```
+
+Key settings include the density threshold, per-task throughput cap, parallel thread count, and whether to auto-stop once balanced.
+
+## Benefits for operators
Review Comment:
Before this add :
```suggestion
### Configuration Parameters
The following parameters can be specified during **start** or **update configuration** Disk Balancer:
| Parameter | Short Flag | Default Value | Description |
| --------- | ---------- |---------------| ----------- |
| `--threshold` | `-t` | `10.0` | Percentage deviation from average utilization of the disks after which a datanode will be rebalanced. |
| `--bandwidth-in-mb` | `-b` | `10` | Maximum bandwidth for DiskBalancer per second. |
| `--parallel-thread` | `-p` | `5` | Max parallel thread count for DiskBalancer. |
| `--stop-after-disk-even` | `-s` | `true` | Stop DiskBalancer automatically after disk utilization is even. |
## Benefits for operators
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739929557
##########
blog/2026-01-29-disk-balancer-preview.md:
##########
@@ -0,0 +1,103 @@
+---
+title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
+authors: ["apache-ozone-community","jojochuang", "0lai0","Gargi-jais11"]
+date: 2026-01-29
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+Ever replaced a drive on a Datanode only to watch it become an I/O hotspot?
+Or seen one disk hit 95% usage while others on the same machine sit idle?
+These imbalances create performance bottlenecks and increase failure risk.
+Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode. To use it, first enable the feature by setting `hdds.datanode.disk.balancer.enabled = true` in `ozone-site.xml` on your Datanodes. Once enabled, clients use `ozone admin datanode diskbalancer` commands to talk directly to Datanodes, with SCM only used to discover IN_SERVICE datanodes when running batch operations with `--in-service-datanodes`.
+
+### Disk Balancing Policies
Review Comment:
Let's change it to :
```suggestion
## How DiskBalancer Decides What to Move
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3783518286
I couldn't find the exact publication date, so I used the file creation date(2025/12/9) instead. If there is a correct publication date, please let me know and I'll update it. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739937500
##########
blog/2026-01-29-disk-balancer-preview.md:
##########
@@ -0,0 +1,103 @@
+---
+title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
+authors: ["apache-ozone-community","jojochuang", "0lai0","Gargi-jais11"]
+date: 2026-01-29
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+Ever replaced a drive on a Datanode only to watch it become an I/O hotspot?
+Or seen one disk hit 95% usage while others on the same machine sit idle?
+These imbalances create performance bottlenecks and increase failure risk.
+Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode. To use it, first enable the feature by setting `hdds.datanode.disk.balancer.enabled = true` in `ozone-site.xml` on your Datanodes. Once enabled, clients use `ozone admin datanode diskbalancer` commands to talk directly to Datanodes, with SCM only used to discover IN_SERVICE datanodes when running batch operations with `--in-service-datanodes`.
+
+### Disk Balancing Policies
Review Comment:
Sure, that's pretty good. Thanks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739881916
##########
blog/2026-01-29-disk-balancer-preview.md:
##########
@@ -0,0 +1,103 @@
+---
+title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
+authors: ["apache-ozone-community"]
+date: 2026-01-29
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+Ever replaced a drive on a Datanode only to watch it become an I/O hotspot?
+Or seen one disk hit 95% usage while others on the same machine sit idle?
+These imbalances create performance bottlenecks and increase failure risk.
+Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode. To use it, first enable the feature by setting `hdds.datanode.disk.balancer.enabled = true` in `ozone-site.xml` on your Datanodes. Once enabled, clients use `ozone admin datanode diskbalancer` commands to talk directly to Datanodes, with SCM only used to discover IN_SERVICE datanodes when running batch operations with `--in-service-datanodes`.
+
+### Disk Balancing Policies
+
+DiskBalancer uses simple but robust policies to decide **which disks to balance** and **which containers to move** (see the design doc for details: `diskbalancer.md` in [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)).
+
+- **Default Volume Choosing Policy**: Picks the most over‑utilized volume as the source and the most under‑utilized volume as the destination, based on each disk’s **Volume Data Density** and the Datanode’s average utilization.
+- **Default Container Choosing Policy**: Scans containers on the source volume and moves only **CLOSED** containers that are not already being moved. To avoid repeatedly scanning the same list, it caches container metadata with automatic expiry.
+
+These defaults aim to make safe, incremental moves that converge the disks toward an even utilization state.
+
+### Container Move Process
+
+When DiskBalancer moves a container from one disk to another on the **same Datanode**, it follows a careful copy‑and‑import flow (summarized from the design doc for [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)):
Review Comment:
```suggestion
When DiskBalancer moves a container from one disk to another on the **same Datanode**, it follows a careful **"Copy-Validate-Replace"** flow (summarized from the design doc for [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)):
```
##########
blog/2026-01-29-disk-balancer-preview.md:
##########
@@ -0,0 +1,103 @@
+---
+title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
+authors: ["apache-ozone-community"]
+date: 2026-01-29
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+Ever replaced a drive on a Datanode only to watch it become an I/O hotspot?
+Or seen one disk hit 95% usage while others on the same machine sit idle?
+These imbalances create performance bottlenecks and increase failure risk.
+Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode. To use it, first enable the feature by setting `hdds.datanode.disk.balancer.enabled = true` in `ozone-site.xml` on your Datanodes. Once enabled, clients use `ozone admin datanode diskbalancer` commands to talk directly to Datanodes, with SCM only used to discover IN_SERVICE datanodes when running batch operations with `--in-service-datanodes`.
+
+### Disk Balancing Policies
+
+DiskBalancer uses simple but robust policies to decide **which disks to balance** and **which containers to move** (see the design doc for details: `diskbalancer.md` in [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)).
+
+- **Default Volume Choosing Policy**: Picks the most over‑utilized volume as the source and the most under‑utilized volume as the destination, based on each disk’s **Volume Data Density** and the Datanode’s average utilization.
+- **Default Container Choosing Policy**: Scans containers on the source volume and moves only **CLOSED** containers that are not already being moved. To avoid repeatedly scanning the same list, it caches container metadata with automatic expiry.
+
+These defaults aim to make safe, incremental moves that converge the disks toward an even utilization state.
+
+### Container Move Process
+
+When DiskBalancer moves a container from one disk to another on the **same Datanode**, it follows a careful copy‑and‑import flow (summarized from the design doc for [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)):
+
+1. Create a temporary copy of the CLOSED container on the destination disk.
+2. Transition that copy into a **RECOVERING** state and import it as a new container on the destination.
+3. Once import and metadata updates succeed, delete the original CLOSED container from the source disk.
+
+This ensures that data is always consistent: the destination copy is fully validated before the original is removed, minimizing risk during balancing.
+
+## Using Disk Balancer
+
+First, enable the Disk Balancer feature on each Datanode by setting the following in `ozone-site.xml`:
+
+- `hdds.datanode.disk.balancer.enabled = true`
+
+CLI examples:
+
+```bash
+# Start DiskBalancer on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer start --in-service-datanodes
+
+# Stop DiskBalancer on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer stop --in-service-datanodes
+
+# Check DiskBalancer status on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer status --in-service-datanodes
+
+# Get a volume density report to see imbalance across disks
+ozone admin datanode diskbalancer report --in-service-datanodes
+
+# Update DiskBalancer configuration on all IN_SERVICE datanodes
+ozone admin datanode diskbalancer update --in-service-datanodes
+```
+
+Key settings include the density threshold, per-task throughput cap, parallel thread count, and whether to auto-stop once balanced.
Review Comment:
DiskBalancer commands are direct from c;lient to datanode. so it can run by giving specific dns hostname or ipaddress as well. Update this section with below contents.
```suggestion
## Using Disk Balancer
First, enable the Disk Balancer feature on each Datanode by setting the following in `ozone-site.xml`:
- `hdds.datanode.disk.balancer.enabled = true`
The Disk Balancer CLI supports two command patterns:
- `ozone admin datanode diskbalancer <command> --in-service-datanodes` - Operate on all **IN_SERVICE and HEALTHY** datanodes
- `ozone admin datanode diskbalancer <command> <dn-hostname/dn-ipaddress:port>` - Operate on a specific datanode
Available commands:
- **start** - Start the Disk Balancer on the target datanode(s)
- **stop** - Stop the Disk Balancer on the target datanode(s)
- **status** - Check the current Disk Balancer status
- **report** - Get a volume density report showing imbalance across disks
- **update** - Update Disk Balancer configuration settings
Examples:
```bash
# Start Disk Balancer
ozone admin datanode diskbalancer start --in-service-datanodes
or
ozone admin datanode diskbalancer start ozone-datanode-1 ozone-datanode-5
# user can also specifiy configuration parameters during start
ozone admin datanode diskbalancer start -t <value> -b <value> -p <value> -s <value> --in-service-datanodes
or
ozone admin datanode diskbalancer start -t <value> -b <value> -p <value> -s <value> ozone-datanode-1
# Stop Disk Balancer
ozone admin datanode diskbalancer stop --in-service-datanodes
or
ozone admin datanode diskbalancer stop 192.168.1.100:9860
# Check status
ozone admin datanode diskbalancer status --in-service-datanodes
or
ozone admin datanode diskbalancer status ozone-datanode-1
# Get volume density report
ozone admin datanode diskbalancer report --in-service-datanodes
or
ozone admin datanode diskbalancer report 192.168.1.100:9860
# Update configuration
ozone admin datanode diskbalancer update -t <value> -b <value> -p <value> -s <value> --in-service-datanodes
or
ozone admin datanode diskbalancer update -t <value> -b <value> -p <value> -s <value> ozone-datanode-1
```
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2734806345
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
Review Comment:
> title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
+1 to this. I find first title as the catchy one as it shows the pain point (“No More Hotspots”) — instantly relatable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3815202736
Thanks @jojochuang and @Gargi-jais11 for the feedback. I had modify outdated context. Please take a look, If there are incorrect contexts, please let me know and I'll update it. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739403679
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
Review Comment:
<img width="2878" height="1522" alt="image" src="https://github.com/user-attachments/assets/b9dac9f1-f71d-43d7-bd8f-b93623d9cf21" />
Will this title look too long on the page?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2735713630
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
Review Comment:
+1 LGTM, This title is quite appealing. Thanks @jojochuang!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2734798557
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
Review Comment:
Yes, please! I think adding @0lai0 and **me** as authors, while also linking or mentioning **the Apache Ozone Community**, is the best way to recognize both the documentation effort and the collective work that went into Diskbalancer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2733761311
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+**Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
Review Comment:
Replace with a better hook.
```suggestion
Ever replaced a drive on a Datanode only to watch it become an I/O hotspot?
Or seen one disk hit 95% usage while others on the same machine sit idle?
These imbalances create performance bottlenecks and increase failure risk.
Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically.
```
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+**Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode; SCM just receives reports.
+
+## Using Disk Balancer
+
+CLI examples:
+
+```bash
+# Start on all datanodes
+ozone admin datanode diskbalancer start -a
+
+# Check status
+ozone admin datanode diskbalancer status
+```
+
+Key settings include the density threshold, per-task throughput cap, parallel thread count, and whether to auto-stop once balanced.
+
+## Benefits for operators
+
+- **Even I/O load** across disks → more stable performance.
+- **Smooth ops after hardware changes** (new or replaced disks).
+- **Hands-off balancing** once enabled.
+- **Clear metrics** for observability and troubleshooting.
+
+It complements the existing Container Balancer: one works across nodes, the other within nodes.
+
+## Closing Thoughts
+
+Disk Balancer is small but impactful. It brings Ozone closer to being a fully self-healing, self-balancing object store — reducing hotspots, simplifying maintenance, and improving cluster longevity.
+
+Ozone 2.2 will ship with this feature enabled via simple CLI controls and safe defaults. If you run long-lived clusters, this is a feature to watch.
Review Comment:
```suggestion
Ozone 2.2 will ship with this feature available via simple CLI controls and safe defaults. If you run long-lived clusters, this is a feature to watch.
```
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
Review Comment:
Let's work on the title a bit more to make it viral.
```suggestion
title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone"
```
```suggestion
title: "Self-Healing Storage: A First Look at Ozone's Datanode Disk Balancer"
```
```suggestion
title: "Your Ozone Datanodes Just Got Smarter: Introducing the Automatic Disk Balancer in Apache Ozone"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739644570
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
Review Comment:
Thanks @jojochuang ! It's my honor. Do I need to modify blog/authors.yaml in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2716407325
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
Review Comment:
This feature is now in apache/ozone master so need of first line.
```suggestion
**Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739639809
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode; SCM just receives reports.
+
+## Using Disk Balancer
+
+CLI examples:
+
+```bash
+# Start on all datanodes
+ozone admin datanode diskbalancer start -a
+
+# Check status
+ozone admin datanode diskbalancer status
+```
+
Review Comment:
Sure, I'll modify to new CLI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3815514838
@ChenSammi Please take a look on the blog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3815361407
ok looks good to me. once the author info is updated we can merge & publish it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3815513072
@0lai0 Please check the UI for Blog. Its broken and messed up. for the last two section.
<img width="1478" height="860" alt="Screenshot 2026-01-29 at 10 36 32 AM" src="https://github.com/user-attachments/assets/bf0d2e8d-2c83-49e0-be57-ede73cefdc82" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3783704333
Thanks @0lai0 for working on this. FYI, till now there is no blog published for DiskBalancer.
The contents of this doc https://docs.google.com/document/d/1hxOZYZ97wNgzbWb0dygVvjUC-cC3RnahB-C6GRn8Y60/edit?tab=t.0#heading=h.rxs6rcvswuwy is outdated.
> This runs independently on each Datanode; SCM just receives reports.
DN doesn't report to SCM about the disk balancer status now. Using CLI user can get the status directly from Datanode. Now SCM has been removed from between. All CLI commands are sent directly from user to DN.
You can refer to the below links to update this blog section for diskbalancer:
1. https://github.com/apache/ozone/blob/master/hadoop-hdds/docs/content/design/diskbalancer.md
2. https://docs.google.com/document/d/1G3fcXqmyiB7MNs7eq0nc4zGmGuLsY_IaFOMpTT5eiq4/edit?tab=t.0#heading=h.goyg6knax8ir
3. https://ozone.apache.org/docs/edge/feature/diskbalancer.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2716402166
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode; SCM just receives reports.
+
+## Using Disk Balancer
Review Comment:
Under this please first specify one need to enable diskBalancer feature flag to use it and then give examples for each start, stop, update, report and status commands for diskbalancer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #276:
URL: https://github.com/apache/ozone-site/pull/276#discussion_r2716335713
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode; SCM just receives reports.
Review Comment:
FYI, SCM is now no longer used for diskBalancer reports. All CLI commands are sent directly from clients to DN.
##########
blog/2025-12-09-disk-balancer-preview.md:
##########
@@ -0,0 +1,68 @@
+---
+title: "Disk Balancer in Apache Ozone: A Preview"
+authors: ["apache-ozone-community"]
+date: 2025-12-09
+tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode]
+---
+
+The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed.
+
+<!-- truncate -->
+
+Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear.
+
+Disk Balancer closes that gap.
+
+## Why Disk Balancer?
+
+- **Disks fill unevenly** when nodes gain or lose volumes.
+
+- **Large deletes** can empty some disks disproportionately.
+
+- **Hot disks degrade performance** and become failure risks.
+
+Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically.
+
+## How it works
+
+The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing.
+
+Balancing is local and safe:
+
+- Only **closed containers** are moved.
+- Moves happen entirely **within the same Datanode.**
+- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks.
+- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O.
+
+This runs independently on each Datanode; SCM just receives reports.
+
+## Using Disk Balancer
+
+CLI examples:
+
+```bash
+# Start on all datanodes
+ozone admin datanode diskbalancer start -a
+
+# Check status
+ozone admin datanode diskbalancer status
+```
+
Review Comment:
These are outdated commands. Please use the links provided to update the new CLI commands. And add all of them over here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3783734782
Add more information about DiskBalancer like add a section describing `DiskBalancing Policies` and `Container Move Process`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] Blog: Disk Balancer in Apache Ozone: A Preview [ozone-site]
Posted by "0lai0 (via GitHub)" <gi...@apache.org>.
0lai0 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3809959236
Thanks @jojochuang and @Gargi-jais11 for the assistance and comments. I'll integrate the feedback and push the changes. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14324.[Website v2] [Blog] Introducing the Automatic Disk Balancer in Apache Ozone [ozone-site]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #276:
URL: https://github.com/apache/ozone-site/pull/276#issuecomment-3815489429
@0lai0 Thanks for the patch. Mostly LGTM. Once the above comments are fixed its good to go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org