You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jojochuang (via GitHub)" <gi...@apache.org> on 2025/06/08 05:39:21 UTC
[PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
jojochuang opened a new pull request, #8585:
URL: https://github.com/apache/ozone/pull/8585
## What changes were proposed in this pull request?
HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes.
Please describe your PR in detail:
* Draft was generated by Copilot Agent (Preview) using prompt:
```
I tried to find any reference to volumes in an Ozone Datanode, but couldn't find any.
https://ozone.apache.org/docs/edge/concept/datanodes.html
I want to add a section "Volume management" in Datanodes.md
We should
(1) explain what a volume is in the context of Ozone DataNode; do not confuse it with namespace volumes in Ozone Manager. That a "volume" is essentially a disk, and a volume stores many containers.
(2) I wanted to also add descriptions for volume choosing policy. Find the description for the property hdds.datanode.volume.choosing.policy. There are two policies: CapacityVolumeChoosingPolicy (default) and RoundRobinVolumeChoosingPolicy. Use the source code as the source of truth to explain what they do and how they work.
(3) Create a table to include configuration properties related to volumes. Include default values and descriptions.
(3) Add a disk balancer description – while volume choosing policy is useful, cluster operations can result in imbalanced disks. For example, in scenarios that we need to add new disks or to replace broken disks, the datanode's disks usage becomes uneven Ozone community is developing a new feature 'disk balancer' that aims to balance disks of Datanodes. The feature is under development of [HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713), and is coming soon.
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-13208
## How was this patch tested?
User doc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #8585:
URL: https://github.com/apache/ozone/pull/8585#discussion_r2157959106
##########
hadoop-hdds/docs/content/concept/Datanodes.md:
##########
@@ -77,6 +77,40 @@ This extra indirection helps tremendously with scaling Ozone. SCM has far
less block data to process and the namespace service (Ozone Manager) as a
different service are critical to scaling Ozone.
+## Volume Management
+
+### What is a Volume?
+
+In the context of an Ozone DataNode, a "volume" refers to a physical disk or storage device managed by the DataNode. Each volume can store many containers, which are the fundamental units of storage in Ozone. This is different from the "volume" concept in Ozone Manager, which refers to a namespace for organizing buckets and keys.
+
+The status of volumes, including used space, available space and whether or not they are operational or removed, can be looked up from DataNode Web UI.
Review Comment:
removed - > failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #8585:
URL: https://github.com/apache/ozone/pull/8585#discussion_r2157959106
##########
hadoop-hdds/docs/content/concept/Datanodes.md:
##########
@@ -77,6 +77,40 @@ This extra indirection helps tremendously with scaling Ozone. SCM has far
less block data to process and the namespace service (Ozone Manager) as a
different service are critical to scaling Ozone.
+## Volume Management
+
+### What is a Volume?
+
+In the context of an Ozone DataNode, a "volume" refers to a physical disk or storage device managed by the DataNode. Each volume can store many containers, which are the fundamental units of storage in Ozone. This is different from the "volume" concept in Ozone Manager, which refers to a namespace for organizing buckets and keys.
+
+The status of volumes, including used space, available space and whether or not they are operational or removed, can be looked up from DataNode Web UI.
Review Comment:
removed - > failed.
operational- > operational(healthy)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #8585:
URL: https://github.com/apache/ozone/pull/8585
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #8585:
URL: https://github.com/apache/ozone/pull/8585#issuecomment-3031677392
Merged. Thanks @peterxcli @chungen0126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #8585:
URL: https://github.com/apache/ozone/pull/8585#discussion_r2157961328
##########
hadoop-hdds/docs/content/concept/Datanodes.md:
##########
@@ -77,6 +77,40 @@ This extra indirection helps tremendously with scaling Ozone. SCM has far
less block data to process and the namespace service (Ozone Manager) as a
different service are critical to scaling Ozone.
+## Volume Management
Review Comment:
@jojochuang , thanks for working on this document improvement.
Currently there are three types of volumes on Datanode, data volume, metadata volume and DB volume. All these new contents are about datanode volume. So either we change the title to "Data Volume Management", or add missing contents for metadata volume and DB volume.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #8585:
URL: https://github.com/apache/ozone/pull/8585#discussion_r2157961328
##########
hadoop-hdds/docs/content/concept/Datanodes.md:
##########
@@ -77,6 +77,40 @@ This extra indirection helps tremendously with scaling Ozone. SCM has far
less block data to process and the namespace service (Ozone Manager) as a
different service are critical to scaling Ozone.
+## Volume Management
Review Comment:
@jojochuang , thanks for working on this document improvement.
Currently there are three type of volumes on Datanode, data volume, metadata volume and DB volume. All these new contents are about datanode volume. So either we change the title to "Data Volume Management", or add more contents for metadata volume and DB volume.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #8585:
URL: https://github.com/apache/ozone/pull/8585#issuecomment-3020648181
Updated. please review again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #8585:
URL: https://github.com/apache/ozone/pull/8585#discussion_r2182265848
##########
hadoop-hdds/docs/content/concept/Datanodes.md:
##########
@@ -88,3 +122,4 @@ hdds.datanode.dir | none | Determines where HDDS data will be stored on the loca
hdds.datanode.dir.du.reserved | none | Reserved space in bytes per volume. Always leave this much space free for non dfs use.
ozone.metadata.dirs | none | Directory to store persisted data (RocksDB).
ozone.recon.address | 0.0.0.0:9891 | RPC address of the Recon. Use <host:port> to connect Recon.
+
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13208. [Docs] Add volume management section under Architecture/Datanodes. [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #8585:
URL: https://github.com/apache/ozone/pull/8585#discussion_r2157961328
##########
hadoop-hdds/docs/content/concept/Datanodes.md:
##########
@@ -77,6 +77,40 @@ This extra indirection helps tremendously with scaling Ozone. SCM has far
less block data to process and the namespace service (Ozone Manager) as a
different service are critical to scaling Ozone.
+## Volume Management
Review Comment:
@jojochuang , thanks for working on this document improvement.
Currently there are three types of volumes on Datanode, data volume, metadata volume and DB volume. All these new contents are about datanode volume. So either we change the title to "Data Volume Management", or add more contents for metadata volume and DB volume.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org