You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "kerneltime (via GitHub)" <gi...@apache.org> on 2025/03/21 17:20:57 UTC
[PR] Hdds 11938. Add overview/docs landing page [ozone-site]
kerneltime opened a new pull request, #136:
URL: https://github.com/apache/ozone-site/pull/136
## What changes were proposed in this pull request?
- Add comprehensive 'What is Ozone?' section focusing on modern data lake and AI storage use cases
- Organize features into logical categories with detailed descriptions
- Add architecture overview section explaining key components
- Include storage elements section explaining the hierarchy
- Add getting started section with link to quick start guide
Fixes: HDDS-9864
This message highlights the key improvements while emphasizing the modern positioning of Ozone for data lakes and AI workloads rather than just Hadoop integration.
## What is the link to the Apache Jira?
https://issues.apache.org/jira/browse/HDDS-9864
## How was this patch tested?
Check off which of the following tests were done on this change. If additional testing was done, please elaborate here as well.
- [X] The CI checks on my fork are passing
- [X] I verified the rendered content using a local preview
- [ ] I manually verified the steps provided in this change work as described
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11938. Add overview/docs landing page [ozone-site]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #136:
URL: https://github.com/apache/ozone-site/pull/136#discussion_r2008404876
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
+
+Ozone organizes storage in a three-level hierarchy:
Review Comment:
We should add a link to the namespace overview page somewhere in this section for more information too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11938. Add overview/docs landing page [ozone-site]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime closed pull request #136: HDDS-11938. Add overview/docs landing page
URL: https://github.com/apache/ozone-site/pull/136
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11938. Add overview/docs landing page [ozone-site]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #136:
URL: https://github.com/apache/ozone-site/pull/136#discussion_r2008373133
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
Review Comment:
For each protocol listed here, we should link to its section in the docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11938. Add overview/docs landing page [ozone-site]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #136:
URL: https://github.com/apache/ozone-site/pull/136#discussion_r2008374572
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
Review Comment:
```suggestion
- **Strong Authentication**: Integrates with [Kerberos authentication](administrator-guide/configuration/security/kerberos) for robust security
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
Review Comment:
For each protocol listed here, we should link to it section in the docs.
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
Review Comment:
```suggestion
- **Encryption**: Transparent data encryption [at rest](administrator-guide/configuration/security/encryption/transparent-data-encryption) and [in-flight](administrator-guide/configuration/security/encryption/network-encryption) to protect sensitive information
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
Review Comment:
This should come before the architecture section since it's higher level, and architecture references these concepts.
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
Review Comment:
This section is pretty random, although the individual points are valid. Can we regroup them elsewhere? EC/replication and fault tolerance probably belong under "Robust Data Management". Observability could be its own section.
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
+
+Ozone organizes storage in a three-level hierarchy:
+
+- **Volumes**: Similar to accounts, created by administrators for organizations or teams
+- **Buckets**: Created by users within volumes, similar to S3 buckets
+- **Keys**: Data objects stored inside buckets, each potentially containing multiple blocks
+
+When a client writes data, Ozone stores it on Datanodes in chunks called blocks, which are organized into containers for efficient management and replication.
+
+## Getting Started
+
+To get started with Ozone, see the [Quick Start Guide](./02-quick-start/01-installation/01-docker.md) for installation instructions and basic usage examples.
Review Comment:
Update this PR with the latest feature branch to get a landing page for each subsection. Also do not include the numbered prefixes in links since they'll break (and fail the build) if sections are changed or moved. Docusaurus will resolve the links without them.
```suggestion
To get started with Ozone, see the [Quick Start Guide](quick-start) for installation instructions and basic usage examples.
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
Review Comment:
```suggestion
Apache Ozone is a scalable, reliable, distributed storage system optimized for data analytics and object store workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently scales to petabytes of data and billions of objects while managing both small and large files.
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
Review Comment:
```suggestion
As a modern storage solution for data lakes and AI workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications like Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
Review Comment:
```suggestion
- **Fine-Grained Authorization**: Support for both native ACLs and [Apache Ranger integration](administrator-guide/configuration/security/ranger) for centralized authorization policies
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
Review Comment:
I'm not sure most readers would connect the dots as to what the actual implications of this are, so might be better to call out explicitly:
```suggestion
- **Separation of Namespaces**: Decouples namespace management from block space management, allowing the cluster to scale with capacity regardless of file sizes
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
Review Comment:
```suggestion
Ozone has a layered architecture that separates namespace management from block space management:
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
Review Comment:
```suggestion
This separation allows Ozone to achieve the scale required of modern storage systems while maintaining high performance and reliability.
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
Review Comment:
```suggestion
- **Datanodes**: Store the actual data in storage containers and provide read and write access
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
+
+Ozone organizes storage in a three-level hierarchy:
Review Comment:
```suggestion
Ozone organizes data in a three-level hierarchy:
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
Review Comment:
```suggestion
- **Storage Container Manager (SCM)**: Manages storage containers which contain block data and handles block allocation
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
+
+Ozone organizes storage in a three-level hierarchy:
+
+- **Volumes**: Similar to accounts, created by administrators for organizations or teams
+- **Buckets**: Created by users within volumes, similar to S3 buckets
+- **Keys**: Data objects stored inside buckets, each potentially containing multiple blocks
Review Comment:
```suggestion
- **Keys**: Data objects stored inside buckets comprised of multiple blocks
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
+
+Ozone organizes storage in a three-level hierarchy:
+
+- **Volumes**: Similar to accounts, created by administrators for organizations or teams
Review Comment:
```suggestion
- **Volumes**: Similar to tenants, created by administrators for organizations or teams
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
Review Comment:
```suggestion
## Namespace Layout
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
Review Comment:
I would combine this with the multi-protocol section
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
+- **Metadata Management**: Efficient handling of metadata with dedicated services for high performance
+- **Snapshots**: Support for point-in-time snapshots to protect against data corruption and facilitate backups
+
+### Operational Excellence
+
+- **Fault Tolerance**: Designed for resiliency with automatic recovery mechanisms to handle failures at all levels
+- **Observability**: Comprehensive metrics, logging, and monitoring through web UI, Prometheus integration, and Grafana dashboards
+- **Replication and Erasure Coding**: Flexible data protection strategies to balance storage efficiency and durability requirements
+
+### Integration with Data Analytics Ecosystem
+
+- **Hadoop Ecosystem**: Seamless integration with Hadoop, Hive, and Spark workloads
+- **SQL Engines**: Works with SQL query engines like Hive, Impala, and Trino without modification
+- **Modern Data Formats**: Supports modern table formats like Apache Iceberg for data lake architectures
+
+## Architecture Overview
+
+Ozone has a layered architecture that separates namespace management from storage management:
+
+- **Ozone Manager (OM)**: Manages the namespace hierarchy (volumes, buckets, and keys) and handles client metadata operations
+- **Storage Container Manager (SCM)**: Manages the containers where data is stored and handles block allocation
+- **Datanodes**: Store the actual data in containers and provide read and write access
+- **Recon**: Analytics and monitoring service that provides insight into the cluster
+
+This separation allows Ozone to achieve the scale required for modern storage systems while maintaining high performance and reliability.
+
+## Storage Elements
+
+Ozone organizes storage in a three-level hierarchy:
+
+- **Volumes**: Similar to accounts, created by administrators for organizations or teams
+- **Buckets**: Created by users within volumes, similar to S3 buckets
+- **Keys**: Data objects stored inside buckets, each potentially containing multiple blocks
+
+When a client writes data, Ozone stores it on Datanodes in chunks called blocks, which are organized into containers for efficient management and replication.
Review Comment:
This seems better suited for the architecture section. You can use Ozone's namespace layout without knowing this implementation detail.
```suggestion
When a client writes data, Ozone stores it as blocks on the Datanodes, which are organized into storage containers for efficient management and replication.
```
##########
docs/01-overview.md:
##########
@@ -5,8 +5,73 @@ slug: /
# Overview
-**TODO:** [HDDS-9864](https://issues.apache.org/jira/browse/HDDS-9864) complete this page
-
## What is Ozone?
-## Features
+Apache Ozone is a scalable, redundant, and distributed object storage system optimized for big data workloads. Designed to address the limitations of traditional storage systems, Ozone efficiently manages both small and large files alike, scaling to billions of objects of varying sizes.
+
+As a modern storage solution for data lakes and AI/ML workloads, Ozone provides a high-performance foundation that seamlessly integrates with existing big data frameworks. Applications built on Apache Spark, Hive, Hadoop, and other data processing engines work natively with Ozone without modifications.
+
+Ozone combines the best aspects of traditional distributed file systems with cloud-native object storage capabilities, delivering durability, consistency, and performance at scale.
+
+## Key Features
+
+### Scalable Architecture
+
+- **Billions of Objects**: Designed from the ground up to store and manage billions of objects efficiently
+- **Separation of Namespaces**: Decouples namespace management from block space management, allowing independent scaling on both axes
+- **Dense Storage Support**: Optimized for high-density storage nodes with support for up to 400TB per node (compared to 100TB in traditional HDFS)
+
+### Multi-Protocol Support
+
+- **S3 Compatible API**: Native support for the Amazon S3 API, enabling seamless integration with S3-compatible tools and applications
+- **Hadoop Compatible File System**: Provides a filesystem interface that works with existing Hadoop ecosystem applications
+- **Rich API Options**: Supports multiple client interfaces including Java API, command-line tools, and REST endpoints
+
+### Enterprise-Ready Security
+
+- **Strong Authentication**: Integrated Kerberos authentication with robust security mechanisms
+- **Fine-Grained Authorization**: Support for both native ACLs and Apache Ranger integration for centralized authorization policies
+- **Encryption**: Transparent data encryption at rest and in-flight to protect sensitive information
+
+### Robust Data Management
+
+- **Strong Consistency**: Provides strict consistency to simplify application design and ensure data integrity
Review Comment:
Strong or strict? I think there's a technical difference but I don't recall exactly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11938. Add overview/docs landing page [ozone-site]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #136:
URL: https://github.com/apache/ozone-site/pull/136#issuecomment-2752431587
Closing this for now, I think there can be a better version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org