You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Gargi-jais11 (via GitHub)" <gi...@apache.org> on 2026/01/27 06:20:16 UTC
[PR] HDDS-14464. Docs] Core Concepts -> Namespace -> Bucket -> Layouts -> File System Optimized Buckets (FSO) [ozone-site]
Gargi-jais11 opened a new pull request, #294:
URL: https://github.com/apache/ozone-site/pull/294
## What changes were proposed in this pull request?
Added new content for v2 doc page https://ozone-site-v2.staged.apache.org/docs/core-concepts/namespace/buckets/layouts/file-system-optimized
- Overview (What is FSO bucket?)
- Why FSO for Hadoop?
- When to use FSO and OBS?
Also provided link to `07-system-internals/07-features/01-filesystem-optimization.md` for internal implementation.
## What is the link to the Apache Jira?
https://issues.apache.org/jira/browse/HDDS-14464
## How was this patch tested?
Check off which of the following tests were done on this change. If additional testing was done, please elaborate here as well.
- [ ] The CI checks on my fork are passing
- [ ] I verified the rendered content using a local preview
- [ ] I manually verified the steps provided in this change work as described
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14464. Docs] Core Concepts -> Namespace -> Bucket -> Layouts -> File System Optimized Buckets (FSO) [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #294:
URL: https://github.com/apache/ozone-site/pull/294#discussion_r2730871079
##########
docs/03-core-concepts/03-namespace/02-buckets/04-layouts/02-file-system-optimized.md:
##########
@@ -4,4 +4,60 @@ sidebar_label: File System Optimized (FSO)
# File System Optimized Buckets (FSO)
-**TODO:** File a subtask under [HDDS-9857](https://issues.apache.org/jira/browse/HDDS-9857) and complete this page or section.
+## Overview
+
+**File System Optimized** (FSO) is a bucket layout optimized for Hadoop Compatible File System (HCFS) operations. Unlike the Object Store (OBS) layout, FSO maintains separate entries for intermediate directories, enabling efficient file system operations like directory listing, renaming, and deletion.
+FSO buckets support **atomic rename** and **delete operations** on directories at any level in constant time, regardless of directory depth or the number of files contained within.
+
+For example, in an FSO bucket, keys are stored with their hierarchical structure preserved:
+
+```text
+/mybucket/data/2025/nov/report
+/mybucket/data/2025/dec/summary
+/mybucket/archive/2024/logs/applog
+```
+
+Each intermediate directories (`data`, `2025`, `nov`, etc.) are stored as a separate entry, allowing efficient directory-level operations.
+
+:::note
+FSO is the default bucket layout in Ozone. To explicitly specify FSO layout when creating a bucket, use the `--layout` flag:
+
+```bash
+ozone sh bucket create /<volume-name>/<bucket-name> --layout FILE_SYSTEM_OPTIMIZED
+```
+
+:::
+
+## Why FSO for Hadoop?
Review Comment:
```suggestion
## Why FSO for Ozone?
```
##########
docs/03-core-concepts/03-namespace/02-buckets/04-layouts/02-file-system-optimized.md:
##########
@@ -4,4 +4,60 @@ sidebar_label: File System Optimized (FSO)
# File System Optimized Buckets (FSO)
-**TODO:** File a subtask under [HDDS-9857](https://issues.apache.org/jira/browse/HDDS-9857) and complete this page or section.
+## Overview
+
+**File System Optimized** (FSO) is a bucket layout optimized for Hadoop Compatible File System (HCFS) operations. Unlike the Object Store (OBS) layout, FSO maintains separate entries for intermediate directories, enabling efficient file system operations like directory listing, renaming, and deletion.
+FSO buckets support **atomic rename** and **delete operations** on directories at any level in constant time, regardless of directory depth or the number of files contained within.
+
+For example, in an FSO bucket, keys are stored with their hierarchical structure preserved:
+
+```text
+/mybucket/data/2025/nov/report
+/mybucket/data/2025/dec/summary
+/mybucket/archive/2024/logs/applog
+```
+
+Each intermediate directories (`data`, `2025`, `nov`, etc.) are stored as a separate entry, allowing efficient directory-level operations.
+
+:::note
+FSO is the default bucket layout in Ozone. To explicitly specify FSO layout when creating a bucket, use the `--layout` flag:
+
+```bash
+ozone sh bucket create /<volume-name>/<bucket-name> --layout FILE_SYSTEM_OPTIMIZED
+```
+
Review Comment:
consider mentioning configuration property ozone.default.bucket.layout, ozone.client.fs.default.bucket.layout and ozone.s3g.default.bucket.layout to update default bucket layout.
```suggestion
To update the default layout when creating buckets, configure these properties in ozone-site.xml:
1. ozone.default.bucket.layout (default: none): Sets the default layout for all buckets if no layout is specified during creation by the client.
2. ozone.client.fs.default.bucket.layout (default: FILE_SYSTEM_OPTIMIZED): Sets the default layout for buckets created using the OFS client.
3. ozone.s3g.default.bucket.layout (default: OBJECT_STORE): Defines the default layout for buckets created through the S3 API.
```
##########
docs/03-core-concepts/03-namespace/02-buckets/04-layouts/02-file-system-optimized.md:
##########
@@ -4,4 +4,60 @@ sidebar_label: File System Optimized (FSO)
# File System Optimized Buckets (FSO)
-**TODO:** File a subtask under [HDDS-9857](https://issues.apache.org/jira/browse/HDDS-9857) and complete this page or section.
+## Overview
+
+**File System Optimized** (FSO) is a bucket layout optimized for Hadoop Compatible File System (HCFS) operations. Unlike the Object Store (OBS) layout, FSO maintains separate entries for intermediate directories, enabling efficient file system operations like directory listing, renaming, and deletion.
+FSO buckets support **atomic rename** and **delete operations** on directories at any level in constant time, regardless of directory depth or the number of files contained within.
+
+For example, in an FSO bucket, keys are stored with their hierarchical structure preserved:
+
+```text
+/mybucket/data/2025/nov/report
+/mybucket/data/2025/dec/summary
+/mybucket/archive/2024/logs/applog
+```
+
+Each intermediate directories (`data`, `2025`, `nov`, etc.) are stored as a separate entry, allowing efficient directory-level operations.
+
+:::note
+FSO is the default bucket layout in Ozone. To explicitly specify FSO layout when creating a bucket, use the `--layout` flag:
+
+```bash
+ozone sh bucket create /<volume-name>/<bucket-name> --layout FILE_SYSTEM_OPTIMIZED
+```
+
+:::
+
+## Why FSO for Hadoop?
+
+### 1. Atomic Operations (The O(1) Factor)
+
+In a standard Object Store, if you rename a directory containing **1 million files**, the system has to:
+
+- Find all 1 million keys
+- Copy them to a new path string
+- Delete the 1 million old keys
+
+This is **O(n)** operation — the more files you have, the longer it takes.
+
+In FSO, a **rename** is just a metadata pointer update. To rename `/data` to `/archive`, Ozone simply finds the entry for `data` in the `DirectoryTable` and updates its name to `archive`. All the children (the millions of files) stay exactly where they are because they point to the `unique ID` of that directory, not its name.
+
+### 2. Delete operations
+
+Deleting a directory with millions of files is efficient because all child entries share the same parent ID prefix, allowing Ozone to quickly locate and remove them using prefix-based queries, rather than scanning the entire namespace.
+
+## When to Use FSO vs Object Store (OBS)
+
+Choose **File System Optimized (FSO)** when:
+
+- Using Hadoop Compatible File System interfaces
Review Comment:
```suggestion
- Using Hadoop Compatible File System interfaces
- Storing data for analytics workloads (Hive, Spark ...)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14464. Docs] Core Concepts -> Namespace -> Bucket -> Layouts -> File System Optimized Buckets (FSO) [ozone-site]
Posted by "yuriipalam (via GitHub)" <gi...@apache.org>.
yuriipalam commented on PR #294:
URL: https://github.com/apache/ozone-site/pull/294#issuecomment-3805154669
LGTM @Gargi-jais11
There is a type in the PR title, missing bracket `Docs]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14464. [Docs] Core Concepts -> Namespace -> Bucket -> Layouts -> File System Optimized Buckets (FSO) [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #294:
URL: https://github.com/apache/ozone-site/pull/294#issuecomment-3806934372
Merged. Thanks @Gargi-jais11 @yuriipalam
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14464. [Docs] Core Concepts -> Namespace -> Bucket -> Layouts -> File System Optimized Buckets (FSO) [ozone-site]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #294:
URL: https://github.com/apache/ozone-site/pull/294
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org